search

KhronosGroup / glslang

Khronos-reference front end for GLSL/ESSL, partial front end for HLSL, and a SPIR-V generator.
Other
3k stars 828 forks source link

m4x4._m03_m13_m23 causes crash in glslang (float3 subset of float4x4) #1314

Open mmostajab opened 6 years ago

mmostajab commented 6 years ago

I am having a crash in glslangValidator when compiling TerrainVS.hlsl.txt (Please rename it to TerrainVS.hlsl) and ZPrePassVS.hlsl.txt (Please rename it to ZPrePassVS.hlsl). The command line I use to compile the shader is as follows:

.\glslangValidator.exe -D -fhlsl_functionality1 .\TerrainVS.hlsl -o TerrainVS.spv --target-env vulkan1.0 -S vert -e TerrainVS -V100
.\glslangValidator.exe -D -fhlsl_functionality1 .\ZPrePassVS.hlsl -o ZPrePassVS.spv --target-env vulkan1.0 -S vert -e Common_ZPrepassVS -V100
johnkslang commented 6 years ago

It's triggered by using the m._m03_m13_m23 syntax to extract a float3 out of float4x4. It can handle that syntax for individual components or full columns, but not odder subsets than those.

Instead, you can use float3(m.[0][3], m.[1][3], m.[2][3]) or whatever the correct equivalent really is.

(Glslang cannot do that itself, because the expression could turn into an l-value, but the constructor can only make an r-value. So, glslang needs full support for an "l-value subset of matrix" operator.)

There is an existing issue for it, but good to know also a new case of bumping into it.

Glslang tries to report the missing functionality, but something downstream crashes it before the message comes out. That could perhaps be somewhat improved, especially by modifying the SpvBuildLogger, which you might be able to customize for your environment.

johnkslang commented 6 years ago

See also #1240 and #844.