Binary release artifacts for macOS

[kvark]

Would it be possible to include the Linux/macOS builds in the releases?

[jaebaek]

Similar to the Windows build artifact, you can find the build artifact of top-of-tree from https://ci.appveyor.com/project/antiagainst/directxshadercompiler/builds/38687372/job/v5wrr3o4cwh7lako/artifacts.

However, I am not sure we can call it "release" ..

[kvark]

Ok, great. So why not include them in https://github.com/microsoft/DirectXShaderCompiler/releases ?

[pow2clk]

Hi @kvark!

Forgive me for donning and doffing my msft hat throughout this response.

I'm going to divide this into two parts: Linux and MacOS and then spend most of my time talking about Linux.

We test as much as we can using per-change builds and tests that get run whenever anything is changed (even documentation files). That goes a good way toward verifying that nothing is seriously broken. However, there are additional tests that due to practical or legal reasons, we can't upload here and have tested for every change. Instead these tests are run on separate servers in the bowels of at least two prominent contributors to this project (Microsoft and Google). These are used to vet the compiler for inclusion in these github releases and the vulkan SDK respectively. In both cases, these tests are presently performed almost exclusively on windows.

The reason we have release builds is to give those who are uninterested in the bleeding edge features a stable compiler to use reliably. This testing is needed to grant that certainty. We could take whatever the appveyor Linux build produced at the moment of release and dump it into that zip file, but that would give no advantages to pulling from the build artifacts directly. In fact, it would give all the drawbacks of using the release build and none of the advantages. It would be both out of date and minimally tested. Worse, it might be expected to be of the same quality and assurance as the windows components of that release, which could lead to confused and angry customers.

That said, adding this kind of testing for Linux and including it in the release package is not out of the question. There is clearly interest for that beyond this request and we are giving it careful consideration. I cannot promise anything yet, but this bug is as good as any to serve as a way to track that effort. I want to make clear that it's not a small amount of work, but it is of interest. In the meantime, the appveyor artifacts are literally the best tested Linux compiler for HLSL available at the moment they are produced and while I can't give them any official seal of approval, as biased as I may be for all the time I spent on that port, I think it's of good quality and reliable.

Finally, I'll rip the bandaid off as maybe the sole author of the MacOS port can. There is insufficient interest in the MacOS port to come close to justifying the above work. It is not available in any current build artifacts because of lack of support and lack of interest. It's there for whomever wants to build it, but the only reason it's there in the first place is because I was bored and unemployed. I still feel that it stretches the compiler in ways that are relevant to more interesting platforms, but including the results in an official release is almost certain not to happen. I learned and had fun doing the port anyway 😁

[kvark]

Thank you for the greatly detailed response! To give some context, we were thinking if we could validate the produced HLSL shaders on Linux CI, hence I filed this. But we are fine doing this on Windows CI as well, so this is really not a problem for us.

[kuhar]

@kvark If you are interested in DXC binaries tested for SPIR-V code generation, these are included in Vulkan SDK releases and contain files for Windows/Linux/MacOS. This testing is done by Google and LunarG only.

[alecazam]

So I have to install 2gb of Vulkan SDK to get the 1.2MB DXC executable for macOS. And is there an M1 build? Apple makes a Metal compiler available for Windows, so can we get the same support from Microsoft.

[llvm-beanz]

We have shipped Linux binaries, and I've re-titled to track macOS.

[dafedidi]

Hey, I'm looking into this issue because I found it out during my long search to get a DXC -> Metal Shader Convertion (MSC) path. I'm looking into how we can get this done and have a valid DXIL on mac as well (issue below). With the introduction of that tool (MSC), I believe we should find a way to support it. Building DXC on mac is not too hard but the signing part dll is sadly not possible even on custom hardware/servers/CI.

https://github.com/microsoft/DirectXShaderCompiler/issues/6057

[damyanp]

See #6770, where the validation / hash code is being added to the repo which resolves part of the issue here.