Interesting AMD Stream forums posts! (old posts)

I have compiled over time some interesting post links in AMD Stream forums: some explain things at the time not documented, some others interesting code, or projects done or using it, etc..

Here it goes..

Released GPUwareC with GWSDK 0.5.1 test

GPUware C 0.5 test release

The GPUware C compiler allows one to code in a C-like language to construct AMD GPU kernel calls. It allows one to program with AMD's CAL, to produce high performance software, and still code kernels in a high level language. The AMD IL produced by the compiler is very readable, and can be easily modified by hand.

Can LDS reads be "broadcast" within a wavefront"?


LDS : More info requested

Features of Stream SDK 1.2?


Double memory copy in CAL ? What about calCtxResCreate ?

GPU memory architecture?

Measuring HD 4850 performance 1tflop shader

You can have max 128 registers per thread. Number of wavefronts that can be executed on a single SIMD is decided by register usage in your shader (Total registers per SIMD are 64*256).

bursting global reads and global memory bandwidth?

global GPR vs. global data store (another)


As for PV/PS, you cannot turn them off and you really would not want to turn them off as they provide a performance bonus over normal register usage.
Maximum 2D stream dimensions supported is 8192x8192 and 1D dimensions suported is 2^26.

Either you can rearrange data to match these dimensions or you can try changing algorithm to execute data tile-by-tile on GPU (Take a look at out of core MMM in samples/CPP/apps). 4870 is also having the same limitation.

About r7xx arch
Ok,
So there are 163840 registers on the RV770. There are 10 SIMD's, so that gives us 16384 registers per simd, or 16K x 128bit as specified in the Registers per SIMD Core row.
Now, the article states right above the table that there are 64 threads per wavefront. So, 16384 / 64 gives you 256 registers per thread.
If you run a problem domain of 1026 * 1026, assuming 1 thread per location, that gives you 1,052,676 threads that need to be executed.
Divide that by the wavefront size, gives you 16449(must round up) wavefronts that will be spawned by the GPU for this domain.
Now, lets assume that you have 5 registers per thread(which can be determined from KSA disassembly), this lets you run a MAX of (256/5) = 51 wavefronts in parallel per SIMD, or 510 at a time on the GPU.
So this means that you have enough wavefronts to fill up the GPU at least 32 times.

So, assuming that your application gets all of the resources on the chip, this is what you should expect. However, because of other constraints this is the best case scenario and not the average case. So this should give you some idea about what you can do.

Hope this helps. That review article is fairly well done and if you analyze it with a compute mindset you can figure out a lot of things that are docs don't currently specify.
http://www.anandtech.com/printarticle.aspx?i=3341


Using 4870x2
============

Many wasted hours later, I think I've found my problem. I needed to call calCtxIsEventDone() after calling the kernel for each GPU to allow the concurrency to occur. Seems like a messy trick - perhaps the multiGPU paragraph in the user guide could be expanded to mention this?

Take a look at section 2.16.3 of stream computing user guide to see how to use multiple GPUs in single thread. I would suggest to create seperate threads for multiple GPUs as leveraging kernel asynchronous call requires lots of tuning and the call might not be asyncronous in some cases. Take a look at Brook+ sample MonteCarlo_MultiGPU and tutorial MultiGPU.

Measuring CAL time
==================

The correct way to time a CAL kernel is to follow this pattern:

flush
start timer
execute kernel
wait on event
stop timer

As for PV/PS, you cannot turn them off and you really would not want to turn them off as they provide a performance bonus over normal register usage.