search

m4rs-mt / ILGPU

ILGPU JIT Compiler for high-performance .Net GPU programs
http://www.ilgpu.net
Other
1.39k stars 118 forks source link

SPIR-V backend #16

Closed danielscherzer closed 2 years ago

danielscherzer commented 5 years ago

Hi and thanks for a very useful tool! I come from a graphics background and I love that I can debug my kernels (shaders) on the CPU. But after the debugging phase I would still like to have a hardware independent shader that I can feed directly into some graphics API. If I understand correctly, you can only interact with graphics API via the buffer mapping (like in your SharpDX Interop), which seems to introduce some overhead each frame. One alternative way would be to generate SPIR-V output. This would open up the possibility to cross compile the SPIR-V output to HLSL, GLSL (using SPIRV-Cross) or feed it into Vulkan or other compatible APIs. So the cross compiled shader would also run on GPUs from AMD, INTEL, ...

Cheers, Daniel

m4rs-mt commented 5 years ago

Many thanks for your feedback! Of course, I also like the idea of a cross-platform compiler that creates cross-platform GPU programs that are compatible with any vendor :) However, I played around with the Vulkan API for some time and recognized the difference between the kernel model and the shader model. To the best of my knowledge, it turns out that the graphics part of Vulkan does not support the kernel model. This model is only supported via OpenCL (at the moment?). As far as I have understood, this could change in the future. Correct me if I'm wrong, but SPIR-V shaders for OpenCL cannot be compiled at all (or correctly...) on AMD cards with the OpenCL API (currently?). Therefore, the community has already requested OpenCL support to run kernels with the full ILGPU feature set on Intel and AMD hardware.

I would suggest that we support both: OpenCL kernels via OpenCL source code and SPIR-V kernels that are compatible with either the kernel or the shader model. In addition, it might be useful to add support for special LoadShader API methods for Vulkan interop. The good news is that I've already started working on a SPIR-V and a dedicated OpenCL backend. However, I guess I will implement the OpenCL backend first to support generic kernels on Intel and AMD hardware. What do you think?

danielscherzer commented 5 years ago

Thanks for your fast reply! I'm a novice in OpenCL and CUDA. So regretably I have no insights about which models are supported or the platform independence of SPIR-V OpenCL shaders, but your plan sounds good to me. I alos think the OpenCL backend is the better first step in this direction. A master student of mine is investigating the possibility to use C# as a (debuggable) language for writing GLSL shaders and thats why I came in contact with the topic and your tool. I think your tool is really valuable for his research.

Thansk again! Cheers, Daniel

AlexBatsev commented 5 years ago

Hi. Maybe it's too late to recommend anything, but OpenCL implementation by NVidia has so bad flaws that we need to abandon OpenCL approach in our projects alltogether. Eg. the issue with blocking clEnqueueReadBuffer despite CL_FALSE was used for 'blocking'. First mentions of the problem can be found in 2012 (https://community.khronos.org/t/clenqueuereadbuffer-blocking-always/3721), and this problem is still not fixed even for newest GPUs. There are other numerous problems like extra CPU OpenCL thread which is spinning and thus occupies whole CPU core, out-of-resources problem etc. There is strong oppinion that NVidia just sabotages OpenCL implementation in favour of CUDA (https://github.com/SuperElastix/elastix/issues/70#issuecomment-476540535). This all makes usage of OpenCL completely prohibitive for us and I don't see any viable alternative to Vulcan Compute.

n8allan commented 5 years ago

@AlexBatsev , thank you for reporting your experience. Fortunately, since ILGPU already represents an abstraction layer, having an OpenCL backend at least gives access to Intel and AMD GPUs, whereas CUDA can continue to be used for NVidia.

So please, @m4rs-mt , by all means please yes on an OpenCL backend! Thank you for what looks like great work on this project, I look forward to trying it out when I can used it (target hardware on my project has Intel GPUs).

m4rs-mt commented 5 years ago

@AlexBatsev thank you for your feedback 👍 . I definitely agree about the limited (and problematic) OpenCL support of NVidia. However, the generation of SPIR-V code requires additional work on which we want to focus in the near future. However, in my experience, SPIR-V support is still quite "limited" on AMD and some ARM platforms (in terms of unexpected driver bugs...). Consequently, I follow @n8allan's argument that we should still be able to generate OpenCL kernel code. Furthermore, I strongly recommend using the already available CudaAccelerator for NVidia GPUs in favor of the upcoming OpenCL accelerator.

m4rs-mt commented 5 years ago

I have updated the milestone to v0.8 as general SPIR-V support will be available in the future. However, the initial OpenCL functionality requested will be included in v0.7.

m4rs-mt commented 4 years ago

The ILGPU-IR has been refined in terms of structures, arrays and views (in particular). The updated transformation passes prepare the IR in such a way that in the end it will be much easier to create SPIR-V code 🔢

m4rs-mt commented 2 years ago

@danielscherzer thanks again for your feature request. Adding SPIR-V support is currently one of our priorities, as we have concrete plans to implement such a backend to support Vulkan-based (and possibly MoltenVK-based) applications.

We are closing this issue for now and will provide PRs as soon as they are ready.