KhronosGroup / SPIRV-Cross

SPIRV-Cross is a practical tool and library for performing reflection on SPIR-V and disassembling SPIR-V back to high level languages.
Apache License 2.0
2.07k stars 565 forks source link

Crash on converting SPIRV to ESSL #1939

Closed TheNosiriN closed 2 years ago

TheNosiriN commented 2 years ago

Hi, I've been using SPIRV for quite some time now and it was working perfectly fine. I make a shader in Vulkan GLSL then generate ESSL 3.3 (OpenGL ES shading language) from the SPIRV binary, it has been working without problems until I made this compute shader:

#version 460

layout(local_size_x = 1, local_size_y = 1) in;
layout(binding = 0, r32f) uniform image2D colordepthTex;
layout(binding = 1, rgba32f) uniform image2D uvTex;

layout(push_constant) uniform Camera {
    vec2 frame_size;
    mat4 projection;
    mat4 view;
};

layout(std430, binding = 0) buffer Positions {
    vec3 pos_buffer[];
};

vec3 rayDirection(float fieldOfView, vec2 size, vec2 fragCoord) {
    vec2 xy = fragCoord - size / 2.0;
    float z = size.y / tan(radians(fieldOfView) / 2.0);
    return normalize(vec3(xy, -z));
}

vec2 rayBox(vec3 origin, vec3 direction, vec3 cMin, vec3 cMax) {
   vec3 tMin = (cMin - origin) / direction;
   vec3 tMax = (cMax - origin) / direction;
   vec3 t1 = min(tMin, tMax);
   vec3 t2 = max(tMin, tMax);
   float tNear = max(max(t1.x, t1.y), t1.z);
   float tFar = min(min(t2.x, t2.y), t2.z);
   return vec2(tNear, tFar);
}

#define R frame_size

void main() {
    vec2 C = vec2(gl_GlobalInvocationID.xy);
    vec2 uv = (C - R/2.) - min(R.x, R.y);

    vec3 dir = rayDirection(35.0, frame_size, vec2(C));
    vec3 worldDir = ((view) * vec4(dir, 0.0)).xyz;

    vec3 normal;
    float dp = rayBox(vec3(0.0), vec3(0.0), vec3(1.0), vec3(1.0)).y;

    imageStore(colordepthTex, ivec2(C), vec4(1.0));
    imageStore(uvTex, ivec2(C), vec4(dp));
}

I am using the August 30th codebase and using release libraries. The shader makes the SPIRV cross compiler crash at random times, I would insert a new function then it crashes, I would comment out a line and it would crash, random little things would make it crash. Currently the shader above crashes the GLSL compiler.

After running it with WinDbg, I see this call site at the crash point:

0:000> kcn
 # Call Site
00 ntdll!RtlReportCriticalFailure
01 ntdll!RtlpHeapHandleError
02 ntdll!RtlpHpHeapHandleError
03 ntdll!RtlpLogHeapFailure
04 ntdll!RtlpFreeHeapInternal
05 ntdll!RtlFreeHeap
06 ucrtbase!_free_base
07 hxshc!std::_Deallocate
08 hxshc!std::allocator<std::_List_unchecked_const_iterator<std::_List_val<std::_List_simple_types<unsigned int> >,std::_Iterator_base0> >::deallocate
09 hxshc!std::_Hash_vec<std::allocator<std::_List_unchecked_const_iterator<std::_List_val<std::_List_simple_types<unsigned int> >,std::_Iterator_base0> > >::_Assign_grow
0a hxshc!std::_Hash<std::_Uset_traits<unsigned int,std::_Uhash_compare<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int> >,std::allocator<unsigned int>,0> >::_Forced_rehash
0b hxshc!std::_Hash<std::_Uset_traits<unsigned int,std::_Uhash_compare<unsigned int,std::hash<unsigned int>,std::equal_to<unsigned int> >,std::allocator<unsigned int>,0> >::emplace<unsigned int const & __ptr64>
0c hxshc!spirv_cross::CompilerGLSL::emit_op
0d hxshc!spirv_cross::CompilerGLSL::emit_binary_op
0e hxshc!spirv_cross::CompilerGLSL::emit_instruction
0f hxshc!spirv_cross::CompilerGLSL::emit_block_instructions
10 hxshc!spirv_cross::CompilerGLSL::emit_block_chain
11 hxshc!spirv_cross::CompilerGLSL::emit_function
12 hxshc!spirv_cross::CompilerGLSL::emit_function
13 hxshc!spirv_cross::CompilerGLSL::compile
14 hxshc!HXSHC_GLES_SPIRVToBin
15 hxshc!CompileSPIRV
16 hxshc!RunShaderCompiler
17 hxshc!ParseFromArgs
18 hxshc!main
19 hxshc!invoke_main
1a hxshc!__scrt_common_main_seh
1b KERNEL32!BaseThreadInitThunk
1c ntdll!RtlUserThreadStart

I am able to see the values of the data members of the spirv_cross::CompilerGLSL object at the crash point but I don't know how to export it to show here so I will take screenshots of it if needed.

This is the code where I am invoking compile. It's nothing advanced so I think this is sufficient.

spirv_cross::CompilerGLSL glslc(spirv_bin);
spirv_cross::CompilerGLSL::Options options;
options.version = 330;
options.emit_push_constant_as_uniform_buffer = true;
options.es = true;
glslc.set_common_options(options);
try {
    std::cout << "bin size: " << spirv_bin.size() << '\n';
    gles = glslc.compile();
} catch (std::exception& e) {
    CPRINT( "ESSL Transpile failed: " + std::string(e.what()) );
    return false;
}

The size printed is 1173 so the binary is not empty. I do not believe it is a bug that makes it crash with that compute shader, the shader is fairly simple, it might even problem on my end but I do not know where to start to fix this.

Edit: the above compute shader compiles successfully on CompilerHLSL with shader model 6.6

TheNosiriN commented 2 years ago

Just to show that I have compiled a similar compute shader before. This is the same compute shader but with few things changed and it compiles:

#version 460

layout(local_size_x = 1, local_size_y = 1) in;
layout(binding = 0, r32f) uniform image2D colordepthTex;
layout(binding = 1, rgba32f) uniform image2D uvTex;

layout(push_constant) uniform Camera {
    vec2 frame_size;
    mat4 projection;
    mat4 view;
};

layout(std430, binding = 0) buffer Positions {
    vec3 pos_buffer[];
};

vec3 rayDirection(float fieldOfView, vec2 size, vec2 fragCoord) {
    vec2 xy = fragCoord - size / 2.0;
    float z = size.y / tan(radians(fieldOfView) / 2.0);
    return normalize(vec3(xy, -z));
}

vec2 rayBox(vec3 origin, vec3 direction, vec3 cMin, vec3 cMax) {
   vec3 tMin = (cMin - origin) / direction;
   vec3 tMax = (cMax - origin) / direction;
   vec3 t1 = min(tMin, tMax);
   vec3 t2 = max(tMin, tMax);
   float tNear = max(max(t1.x, t1.y), t1.z);
   float tFar = min(min(t2.x, t2.y), t2.z);
   return vec2(tNear, tFar);
}

#define R frame_size

void main() {
    vec2 C = vec2(gl_GlobalInvocationID.xy);
    vec2 uv = (C - R/2.) - min(R.x, R.y);

    vec3 dir = normalize(vec3(uv,1.0));//rayDirection(35.0, frame_size, vec2(C));
    vec3 worldDir = ((view) * vec4(dir, 0.0)).xyz;// * .5+.5;

    vec3 normal;
    float dp = rayBox(vec3(0.0), vec3(0.0), vec3(1.0), vec3(1.0)).y;

    // imageStore(colordepthTex, ivec2(C), vec4(1.0));
    imageStore(uvTex, ivec2(C), vec4(dp));
}

This is the output GLSL:

#version 330 es
layout(local_size_x = 1, local_size_y = 1, local_size_z = 1) in;

layout(binding = 0, std430) buffer Positions
{
    vec3 pos_buffer[];
} _143;

layout(std140) uniform Camera
{
    vec2 frame_size;
    mat4 projection;
    mat4 view;
} _80;

layout(binding = 1, rgba32f) uniform writeonly highp image2D uvTex;
layout(binding = 0, r32f) uniform readonly writeonly highp image2D colordepthTex;

vec2 rayBox(vec3 origin, vec3 direction, vec3 cMin, vec3 cMax)
{
    vec3 tMin = (cMin - origin) / direction;
    vec3 tMax = (cMax - origin) / direction;
    vec3 t1 = min(tMin, tMax);
    vec3 t2 = max(tMin, tMax);
    float tNear = max(max(t1.x, t1.y), t1.z);
    float tFar = min(min(t2.x, t2.y), t2.z);
    return vec2(tNear, tFar);
}

void main()
{
    vec2 C = vec2(gl_GlobalInvocationID.xy);
    vec2 uv = (C - (_80.frame_size / vec2(2.0))) - vec2(min(_80.frame_size.x, _80.frame_size.y));
    vec3 dir = normalize(vec3(uv, 1.0));
    vec3 worldDir = (_80.view * vec4(dir, 0.0)).xyz;
    vec3 param = vec3(0.0);
    vec3 param_1 = vec3(0.0);
    vec3 param_2 = vec3(1.0);
    vec3 param_3 = vec3(1.0);
    float dp = rayBox(param, param_1, param_2, param_3).y;
    imageStore(uvTex, ivec2(C), vec4(dp));
}
HansKristian-Work commented 2 years ago

Are you sure you're not hitting any ABI issues with mismatched headers and such? Every time this has come up that has been the cause. Does it reproduce with the standalone CLI?

TheNosiriN commented 2 years ago

I didn't even think about using the CLI.

I wrote SPIRV binary to a file the tried it using the CLI on release build and it compiles the shader just fine. image

So it's a problem on my end. I downloaded the August 30 build again, rebuilt it completely using the release setting from cmake, copied the headers and built libraries, but it still crashes.

TheNosiriN commented 2 years ago

I tried something else, I commented out the GlslangToSpv (Vulkan GLSL compiler code) part of my code, then I loaded in the spirv binary file that I had already generated using that code then used the spirv_cross::CompilerGLSL on it and it compiled successfully.

But the weird part is that its only when I comment out the code that generates SPIRV binary that it works for any shader now.

HansKristian-Work commented 2 years ago

Are you trying to link against a pre-built library that was built with a different toolchain?

TheNosiriN commented 2 years ago

No. I am building the project using cmake with release config, then I copy the libraries that built straight to my project for linking.

I tried something else. I tried using google's shaderc for the SPIRV compilation part but I also get the same result, it crashes when I use the spirv compiler but it doesn't crash when I comment it out and just read from the previous .spv file

Is it that I can't use a glsl to spirv compiler and spirv cross in the same program? Do you think I should make them seperate programs then output a temporary binary file to pass around?

TheNosiriN commented 2 years ago

After a couple hours of frustration, I tried using the C api library and I found out that everything works just fine with it. I think I'm going to continue with it.