[Backend] DirectX (D3D12, DXIL) Backend

[python3kgae]

Concisely describe the proposed feature I would like to add a DirectX 12 backend to the compiler so that I can use new features which requires shader model >= 6.0.

Describe the solution you'd like (if any) The plan is in 3 steps.

1. Add llvm 15 support which is required for generate DXIL. Code change will be guarded with something like "#ifdef TI_LLVM_15" if llvm 15 require different implementation. CMake option TI_LLVM_15 will be added to control it and OFF by default.

2. Add AOT compile first which could be tested by add c++ directx12 test without backend. DXIL support in llvm 15 is still in early state. It will be a long journey to support all features. CodeGenLLVM will be used as base class for llvm ir generation when codegen. Then some llvm passes will be added to prepare for DXIL generation and finally use DirectX backend in llvm 15 to generate DXIL. A kernel which has more than 1 tasks is a new thing for DXIL, no plan to support this feature at first.

3. Add DirectX 12 backend. No an expert on this part :( Maybe start support for simple cases first.

Additional comments I'm working on DXIL generation on llvm 15. Targeting language other then HLSL to DXIL will help the design of DXIL generation. This is personal work, not official contribution (which requires examine and approve).

[k-ye]

Hi @python3kgae ,

Thanks for proposing this, it looks really attractive!

Add llvm 15 support which is required for generate DXIL.

Note that we are (conservatively) planning to upgrade to LLVM-12. It's probably fine for us to directly go to 15 as well. One thing we've recently noticed, is the installation speed from apt-get for clang-13 (and likely beyond) is super slow. This could potentially hurt our external contributors...

(Ive followed https://apt.llvm.org/ for clang-13 installation, not sure if you have better suggestions here.)

Then some llvm passes will be added to prepare for DXIL generation and finally use DirectX backend in llvm 15 to generate DXIL.

I see. I'm not sure how feasible it is to share the generated LLVM IR between CUDA and DXIL. Hopefully this is not a problem :-) (Taking SPIR-V as an example. Although it claims to support both Vulkan and OpenCL, they use very different Execution Model, making it nontrivial to share the same SPIR-V between these two APIs).

No an expert on this part :( Maybe start support for simple cases first.

If we can have a working codegen, D3D12 runtime API shouldn't be too hard to sort out :-)

[python3kgae]

Note that we are (conservatively) planning to upgrade to LLVM-12. It's probably fine for us to directly go to 15 as well. One thing we've recently noticed, is the installation speed from apt-get for clang-13 (and likely beyond) is super slow. This could potentially hurt our external contributors...

(Ive followed https://apt.llvm.org/ for clang-13 installation, not sure if you have better suggestions here.)

If clang is only used for COMPILE_LLVM_RUNTIME, We can keep using clang-10 first. llvm 15 should be OK to link bitcode generated with llvm 10. And once apt.llvm.org got the issue fixed, we can update clang to higher version.

[ailzhang]

@python3kgae COMPILE_LLVM_RUNTIME isn't the only place that requires clang version 10, I'm afraid that https://github.com/taichi-dev/taichi/blob/master/cmake/TaichiCore.cmake#L253-L288 also requires headers from llvm-10, so existing codebase might need some effort to make it work with llvm-15. But we'd more than happy to provide guidance if you're interested in hacking a prototype for this, how about joining our slack workspace so that we can discuss how a MVP looks like? https://taichicommunity.slack.com/join/shared_invite/zt-14ic8j6no-Fd~wKNpfskXLfqDr58Tddg#/shared-invite/email Thanks!

[python3kgae]

@ailzhang I checked https://github.com/taichi-dev/taichi/blob/master/cmake/TaichiCore.cmake#L253-L288, it is where llvm header and lib are added. it only require llvm version >= 10, llvm 15 doesn't require any change for this part.

I've got taichi repo compiled with llvm 15. Most of the changes are related to the opaque pointer feature which require add type when create LoadInst/GEP. Currently I'm hacking to get the type from llvm type, need help to get the type from taichi type instead of llvm type.

[qiao-bo]

One thing we've recently noticed, is the installation speed from apt-get for clang-13 (and likely beyond) is super slow. This could potentially hurt our external contributors... (Ive followed https://apt.llvm.org/ for clang-13 installation, not sure if you have better suggestions here.)

This could be specific to the nightly package. I tried the release version on a Ubuntu 22.04. The time of apt-get install clang-13 seems ok. But indeed current 20.04 users will have to suffer... Long-term speaking should not be a blocker.