GPU virtualization (and what to expect in VMs)!

GPU hardware based virtualization

Based on my investigation on August 2009..

TODO: post links..
It has the promise of running a Linux VM on Windows and running CUDA on it at full speed for example..

Right now as much as Intel VT and AMD Pacifica(?) bringed support for CPU virtualization for GPU virtualization it needs Intel VT-d and AMD IOMMU hardware support..

It enables also NIC hardware virtualization for example.

Note this I think allows dedicating one device (GPU,NIC) to VM so more than 1 GPU required if wanting GPGPU on both Windows and Linux..

See suporting chipsets
Link
for Intel:
Requires intel chipsets: mainly 4x professional chipsets (Q4x) current x58 and some x38..
Also I think there is a CPUID flag for querying support?.. I think Nehalem supports it..
almost sure you need Intel mainboard!
Would seem promising Asus X58 board altough BIOS has currently bugs..

AMD: currently no shipping..
See AMD virtualization blog for demo at a conference showing AMD chipset supporting iommu and a FireGL/Stream card using GPU dedicated (Manycours 2010 stuff)..


VM software:

Parallels composer extreme 4.0 anounced in late March 2009 jointly with Nvidia SLI multios shipping in May 2009.
Requires certified motherboard (or BIOS?) and Nvidia supporting sli multios (Quadro 3800 or higher (4800,5800))..
currently working only on HP z.. workstation.

Allows running Windows and Linux in parallel with two GPUs (I think you need only one Quadro other can be Geforce)
Demo on youtube showing this!

On Apple Developers Conference in June 2009
Parallels showed prototype VM soft running Apple with Windows VM with full GPU acceleration.. was 3800 GPU currently not shipping on MAC..
see video..

Regarding other VM Nvidia anounced would be working with VMWARE..

Also new Xen supports VT-d so it would work..
A paper testing it shows ATI works good.. but Nvidia doesn't work and should work so seems some protection
Also surprinsingly Intel worked to some extent..
Tested Windows XP and Vista (i.e. XPDM and WDDM drivers) and recent 3DMark 2006..

Theoretically VT-d is supported in ESX (only on it) but no GPU virtualization testing(?).
Would be good on Workstation..

OpenGL and DirectX passtroughs
==============================

For OpenGL you have 2.1 support in Mac in Fusion 3 and Parallels Dekstop 5.0
For Linux and Windows have guest support for 2.1 also in VMWARE Workstation 7 and VirtualBox 3.1..
More VirtualBox 3.1 adds hardware video acceleration for Windows guests (strecthing and color conversion I think..)

I mean OpenGL commands are forwarded to host GPU and have GPU acceleration on it..
Some overhead but in GPU heavy code (pixel shader heavy code ) should be good enough..
Also alotugh more delicated they provide DirectX 9.0 SM 2.0/3.0 support emulated through DirectX to OpenGL translation (think Wine) layers..

Also I remember some DirectX in Windows guests with Windows hosts without this translation and getting DirectX commands so similar to OpenGL stuff.. (i.e. higher perf and correct results..)
Finally WDDM driver with Aero in Workstation 7 (Windows and Linux host) and Parallels 5.0 (Mac host)
If not exist would be good to add..

CUDA and OpenCL passtroughs
==================

A similar solution could be provided to CUDA via passtrough..
Two papers have explorer it:
1.vcuda (Hunan University, CN, at IPDPS 2009) : in case you can't access to the paper there is a nice homepage with benchmarks go it..
2.GViM (Georgia IT)


Other posibilities
==================
Another related result is running CUDA Windows programs on Linux directly in Windows and Mac..

Search folding@home in linux via wine..
I have checked both CAL and CUDA wrappers for Wine running my matmul test (CUDA and CAL) and worked correctly..