Is the HLSL frontend still worth the cost?

[dj2]

With DXC being release, there is a viable HLSL -> SPIR-V conversion path provided for HLSL. DXC is actively supported and seems like it will have a more up to date HLSL flow.

Given GLSLang is the GLSL reference compiler, is the HLSL frontend still beneficial? There is a cost in terms of maintenance required in order to support HLSL (>20k lines of code, >200k lines of test files). The HLSL code isn't all gated on ENABLE_HLSL, there are checks based on the language used, there is code for implicitThis and vec1 which all add complexity.

The added testing of running the suite of HLSL enabled and disabled has a cost on testing and development time.

So, is the HLSL frontend worth the cost? Historically, the answer was yes, but with DXC being open sourced and supported, is that still the case?

[greg-lunarg]

To me, the question is: is ripping out the HLSL frontend worth the cost? As the primary maintainer of glslang, I have not experienced that HLSL puts any noticeable additional burden on the maintenance of the GLSL path. Same for testing compute time.

At some point soon it will probably make sense to deprecate HLSL support to avoid additional bug fixing, but we will probably want to poll the Vulkan community first.

As for spending the time to rip out HLSL, I think there are probably higher priority items at the moment.

[TheMostDiligent]

Please don't drop the HLSL support!

We use GLSLang as the main compiler for both GLSL and HLSL and this has been a very positive experience. Up to a certain point DXC simply could not produce valid SPIRV for many shaders (the situation has certainly improved by a lot), but I still trust GLSLang more than DXC. Besides, DXC and GLSLang produce different SPIRV constructs in certain scenarios, which require different handling. Using just one compiler for both languages is more convenient.

Besides, GLSLang with all SPIRV-Tools is few times smaller than DXC, we build it for all platforms we support including iOS and Android (and even tvOS) to be able to compile HLSL shaders at run-time.

[dj2]

Removing HLSL would have to go through a deprecation period, and probably a major number change for glslang. It isn't a lot of work to remove the bulk of it, there are probably lots of little bits that could be cleaned up afterwards.

I'm not sure I understand why DXC and GLSLang producing different SPIR-V is an issue? What handling do you need to do that is different between the two systems?

[TheMostDiligent]

As a specific example, consider this code:

cbuffer Constants
{
    float4x4 g_WorldViewProj;
};

glslang emits SPIRV as if the following GLSL was written:

uniform Constants // UB.name
{
    float4x4 g_WorldViewProj;
}; // no instance name

DXC emits the byte code that corresponds to the following GLSL:

uniform type_Constants // UB.name
{
    float4x4 g_WorldViewProj;
}Constants; // get_name(UB.id)

We need to consider the differences like that.

I understand that there may be no work to add new features for GLSLang such as SM6.0+ support, which is totally fine. However existing functionality for SM5.1 seems to be very robust and reliable and can be considered as reference. I really don't understand the need to remove it especially if it requires major changes. Why put effort to kill what works great?

[TheMostDiligent]

Besides, like I mentioned and what is probably more important, DXC is only available on Windows and Ubuntu and not available on many other platforms: MacOS, iOS, Android, tvOS to name just a few. Even if it was, it is much larger. GLSLang provides a universal cross-platform path for HLSL compilation.

[TheMostDiligent]

@dj2 if hopefully you also want to add to the equation how removal of HLSL will affect projects that use GLSLang, for us (https://github.com/DiligentGraphics/DiligentEngine), it will completely break Vulkan rendering backend. There is really absolutely no way to replace GLSLang with DXC, so we may have to go as desperate as stop updating GLSLang.

[dj2]

I don't think it could be removed immediately. There would have to be a deprecation period before it's removed to give downstream users time to migrate off. I'm not sure why you'd need to drop GLSLang for compiling GLSL -> SPIR-V, but that's a choice for your project.

For the given example, so the SPIR-V doesn't match up with what your API side is missing? And the program fails, or is it just different? If it fails, I'd suggest filing a bug with the DXC project about the compilation issue.

All code has a cost, if it's being add too or not. There are a lot of ifdefs in the code, tests to maintain. You have to think about adding things to the GLSL side without breaking the HLSL side. So, yes, there isn't a lot changing, but that doesn't mean there isn't a maintenance cost.

As I said previously, I'm not sure I see why the GLSL reference compiler has an HLSL compiler builtin into it.

[greg-lunarg]

... I'm not sure I see why the GLSL reference compiler has an HLSL compiler built into it.

The HLSL parser was put into glslang several years before the DXC open source project was announced. We had a client that was interested in HLSL -> SPIR-V. Putting an HLSL parser into glslang leveraged the existing AST processing and SPIR-V codegen in glslang.

[TheMostDiligent]

You have to think about adding things to the GLSL side without breaking the HLSL side.

I don't quite understand why/how adding things to GLSL would break HLSL parser so often that removing HLSL completely is a justified remedy.

[dj2]

Because the HLSL support isn't just a parser. There is HLSL code scattered all over, through various boolean flags, the HLSL source type or ifdefs. You need to keep track of if you came from HLSL for things like && because in GLSL it short-circuits, but in HLSL it doesn't.

[StarsX]

I'm not sure if DXC works well with OpenGL. At least in my program, HLSL compiled by glslang works with OpenGL, except this missing feature: https://github.com/KhronosGroup/glslang/issues/2538

[arcady-lunarg]

The cost of maintaining the HLSL frontend has, in practice, not been particularly high, especially since new features aren't really being added to it, and the only changes are occasional bugfixes from community members. So I think the answer to this question is "yes, for now".