search

KhronosGroup / SPIRV-LLVM

This project is no longer active. Please join us at
https://github.com/KhronosGroup/SPIRV-LLVM-Translator
Other
261 stars 60 forks source link

Support Vulkan? #202

Open skl131313 opened 7 years ago

skl131313 commented 7 years ago

Not sure if Vulkan is currently supported or not, but it seems only OpenCL is?

yxsamliu commented 7 years ago

Currently the translator only supports OpenCL.

thewilsonator commented 7 years ago

After I'm done getting LDC to support OpenCL SPIR-V then if there is sufficient demand (principally within the D community, but other contributors are ofc welcome) I may consider making a fork of llvm off the latest version, updating this (OpenCL SPIR-V) to the latest llvm version and adding a Vulkan backend to it. Possibly also considering trying to integrate Microsoft's recently open-sourced DirectXShaderCompiler, although it's git history will definitely be bjorked. Will wait till development of it has matured a bit.

If I get around to doing that I will be changing a few things: most noticeably using llvm intrinsics for the OpenCL/Vulkan intrinsics rather than "itanium with extensions" C++ mangling, making it far easier to use with other tools.

keryell commented 7 years ago

@thewilsonator: while working on https://github.com/keryell/triSYCL/ I am looking at leveraging http://alphanew.net/index.php?section=alphanew&site=overview&lang=eng&newsID=111 Perhaps you can use it too?

thewilsonator commented 7 years ago

@keryell Well, it doesn't have releases which makes doing CI difficult, see #201. It is also 105 commits behind KhronosGroup/SPIRV-LLVM 3.6.1 and is on LLVM 3.9.0 as opposed to the final release of that series 3.9.1. It also hasn't received any commits for 11 months.

Hence why I am considering doing it myself. Also I will probably put the lib/SPIRV and tools/llvm-spirv and tools/spirv-tool in their own submodules, to keep things separate and easily identifiable. Will do the same for MS's DirectXShaderCompiler.

dneto0 commented 7 years ago

Please be aware that the dialect of SPIR-V accepted by Vulkan is significantly different from the dialect accepted by OpenCL. For starters, see the validation rules in the SPIR-V spec for "Shader" vs. "Kernel". Also, check extra restrictions in SPIR-V environment spec in the Vulkan spec: https://www.khronos.org/registry/vulkan/specs/1.0-wsi_extensions/html/vkspec.html#spirvenv

thewilsonator commented 7 years ago

Isn't it just SPIR-V core + Vulkan specific capabilities + Vulkan specific operations (a.k.a. "standard library functions")?

dneto0 commented 7 years ago

Yes, but that's a big difference from SPIR-V for OpenCL. You're subtracting out a lot of assumptions made by SPIRV-LLVM assuming it's targeting OpenCL.

Examples:

I urge you to consider all the details first before diving in by writing code.

keryell commented 7 years ago

@thewilsonator: we always appreciate contributions. :-) On the SPIR-V side, the current LLVM library producing SPIR-V should be updated to a real SPIR-V LLVM back-end, if you are looking to some reorganization ideas...

thewilsonator commented 7 years ago

My reorg ideas are

However I won't have any time to do this until August at the earliest.

thewilsonator commented 7 years ago

@dneto0 thanks for the pointers.

Point 1 seems like a good candidate for an intrinsic function to mark the end.

Point 2 doesn't OpenCL have 5? Private, Local, Global, Constant and Generic? Also seems like a good candidate for addrspace qualification. edit: or is this const, volatile, restrict, ???, @? Found them all valid SPIR-V storage classes are

Point 3 I'm less sure of, but it seems that metadata should work here.

Point 4 It is the host language's problem to annotate everything with noalias.

dneto0 commented 7 years ago

@thewilsonator: About storage classes. Yes, OpenCL has 5 (I forgot Generic).

The validity rules are in two places:

  1. Inside the SPIR-V spec itself.
  2. Each client API that uses SPIR-V specifies additional rules in what it calls an "environment spec". https://www.khronos.org/registry/vulkan/specs/1.0-wsi_extensions/html/vkspec.html#spirvenv

There's a matching one for OpenCL.

thewilsonator commented 7 years ago

@keryell I have got a fork of llvm 5.0 with spir and spir64 proper targets (also made a Triple::spirl for logical spirv memory to ease Vulcan support if I get around to it) at https://github.com/thewilsonator/llvm/tree/compute This is about the last point before I begin to change it use use intrinsics instead of mangled C++, and thus break compatibility with https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 (although the upgrade from 3.6.1 to 5.0 will most definitely have broken things). Let me know if you (or others) want to collaborate on this, more eyes hands and bug reporters are always good.

yxsamliu commented 7 years ago

I am OK with that. Thanks for the efforts.

keryell commented 7 years ago

@thewilsonator For now I am busy on my SYCL device compiler with 3.9, SPIR and SPIR-V. So, along with other projects too, I do not have the bandwidth for more... :-( But at some point I hope there will be some convergence in future versions. :-) Thanks for the work.

thewilsonator commented 7 years ago

Oh well. I'm sure it will be there when you decide you need to update your llvm version. Good luck!

abel-bernabeu commented 7 years ago

Sorry to ask here, but this issue has been open for a while and seems to serve like a forum (given the project lacks an actual forum).

I see the default branch is 3.6, but there is a 3.8 branch which is listed as active. The 3.8 branch does not pass continuous integration, though.

I need to work on fixing bugs on an LLVM related project for gaining familiarity with it and this seems a good place for me to pick up bugs.

Thanks in advance.

abel-bernabeu commented 7 years ago

Ok, I think I figured out this question. The project is based on LLVM 3.6 and that is what the 3.6 branch contains. There was an attempt to upgrade to LLVM 3.8.1 but the tests are not passing with the new version, which is the reason why we are stuck on the 3.6 branch.

Someone has to figure out why the tests are failing with the upgraded version. I could try to look at that then. Tell me if there is anything else you would like me to focus on. For any suggestions, feel free to contact me (abel.bernabeu@gmail.com).

thewilsonator commented 7 years ago

Just an update to all: there is reasonable interest from the LLVM people for upstreaming my work (see http://lists.llvm.org/pipermail/llvm-dev/2017-May/112538.html), which is an updated version of this, as well as some interest from the D community to support Vulkan as part of that.

abel-bernabeu commented 7 years ago

I can agree on most of your suggestions.

However, suggesting that this project should be an LLVM backend is understimating the main attractive of the khronos branch: that it is a bidirectional translator.

It can translate back and forth between LLVM IR and SPIRV because the library keeps the mapping bijective, and that property should be retained for the project to gain traction.

For someone who has a pre-existing frontend and a pre-existing backend, the library allows to use SPIRV as a standard program exchange format.


            llvm-spirv           llvm-spirv (reversing)
            +-----^-----+        +-----^-----+
            |           |        |           |
   source -> ll -> spirv          spirv-> ll -> binary
  |            |                         |           |
  +-----v------+                         +-----v-----+
  Existing frontend                      Existing backend

Is there such a thing as an abstraction of a bidirectional LLVM IR translator on the LLVM framework? I do not think so, but I could be wrong. Having a SPIRV backend is just half of the whole picture.

Ideally you want the concept of bidirectional translator abstracted on an LLVM interface, and then we would stop having SPIRV-LLVM as a library.

thewilsonator commented 7 years ago

that it is a bidirectional translator

It is that at the moment and it will remain one. In fact I hope to improve it by integrating it (not quite the right word) with the other backends i.e. translate the intrinsics. There are some discussions/reservations about the exact implementation but ...

project to gain traction

The last functional commit to this repo was at the start of December...

For someone who has a pre-existing frontend

Thats precisely why I am doing this, see my first post.

and a pre-existing backend, the library allows to use SPIRV as a standard program exchange format

My changes integrate far better with the LLVM machinery and so opens this up to more backends, all that would be required would be to translate the intrinsics. No to mention with stay up to date with LLVM instead of falling 2 years and counting behind.

Ideally you want the concept of bidirectional translator abstracted on an LLVM interface, and then we would stop having SPIRV-LLVM as a library.

That's what llvm-spirv is for, to do the reverse translation. The intrinsic translation will have to be added to it but that requires someone with knowledge of the other backends, who are most likely to be found in the LLVM project itself.

abel-bernabeu commented 7 years ago

Ok, if we still can translate back and forth I guess your fork deserves a try. I would never object to people forking open source stuff :)

Am not an expert in LLVM and I have only started spending o couple hours a week on getting the 3.8 branch on a better shape than it is now. My modest short term goal is getting the continuous integration functional by fixing the current failures. From what I have seen so far some of the issues are due to the tests needing an update but others need fixing the translator.

Apart from that, I did try to manually merge the functional changes from the 3.6 branch that where missing in my working copy of 3.8. The merged tree builds and seems to work, which tells me that, once the continuous integration works, we could start cherry picking from 3.6 into 3.8 trying to keep the history of who made what change.

thewilsonator commented 7 years ago

Neither am I :)

My plan is to: 1) finish moving to an intrinsics/tablegen format. This requires writing a backend for tablegen, and will enabling removing all of the mangling code and custom table stuff. 2) port the tests across from this repo, update them to account for (1) and fix any breakages. 3) make llvm-spirv usable. 4) change the assembly format to match the reference implementation and update the tests accordingly. 5) assuming I haven't forgotten anything upstream it to LLVM.

abergmeier-dsfishlabs commented 7 years ago

I currently have to translate a large codebase from C++11 to Vulkan compute.

I am wondering whether the more sensible approach would be to rather have a transpiler from C++ to SPIRV for Vulkan

The frontend could then:

  1. Handle C++ name resolution
  2. Prevent any call to a mutating function
  3. Prevent access of pointers
  4. Prevent passing of references to calls

The backend would have to:

  1. Handle (naive) name mangling scheme for GLSL

That of course is a oversimplification.

thewilsonator commented 7 years ago

@abergmeier-dsfishlabs I think you would have better luck scripting clang to do the translations of C++ to Vulkan Compute source for you, rather than go via SPIR-V.

If you do decide to go the SPIR-V route https://github.com/google/clspv may be of interest.

fhoenig commented 5 years ago

C++ to Vulkan Spir-V compute shaders is something that I want as well. There is plenty of CUDA source that would justify that. AMD stuff as well.

Vulkan was supposed to bring compute and graphics together no?