search

KhronosGroup / glslang

Khronos-reference front end for GLSL/ESSL, partial front end for HLSL, and a SPIR-V generator.
Other
2.91k stars 817 forks source link

Stackoverflow due to recursive loop #3470

Closed Bardoux closed 5 months ago

Bardoux commented 6 months ago

Hi,

I'm trying to setup a Vulkan environment, and compiling the shaders' code at runtime with the Google's libshaderc. I'm on Ubuntun-mate 23.04, and I use the libshaderc installed with apt.

When running even a quite simple code, I've got a stackoverflow due to a recursive loop on the createSpvConstantFromConstUnionArray method (here).

Here is my code:

#define GLFW_INCLUDE_VULKAN
#include <GLFW/glfw3.h>

#include <iostream>
#include <string>
#include <vector>

#include <cstdlib>

#include <shaderc/shaderc.hpp>

const std::string vertex_name = "vertex_shader";
const std::string vertex_shader = R"(
#version 450

vec2 positions[3] = vec2[](
  vec2(0.0, -0.5),
  vec2(0.5, 0.5),
  vec2(-0.5, 0.5)
);

vec3 colors[3] = vec3[](
  vec3(1.0, 0.0, 0.0),
  vec3(0.0, 1.0, 0.0),
  vec3(0.0, 0.0, 1.0)
);
layout(location = 0) out vec3 fragColor;

void main() {
  gl_Position = vec4(positions[gl_VertexIndex], 0.0, 1.0);
  fragColor = colors[gl_VertexIndex];
}
)";

std::vector<uint32_t> compile_shader(
      const std::string& shader_name,
      shaderc_shader_kind kind,
      const std::string& source,
      bool optimize) {

  shaderc::Compiler       compiler;
  shaderc::CompileOptions options;

  // Like -DMY_DEFINE=1
  // option.AddMacroDefinition("MY_DEFINE", "1");
  if (optimize) {
    options.SetOptimizationLevel(shaderc_optimization_level_size);
  }

  shaderc::SpvCompilationResult module =
      compiler.CompileGlslToSpv(source, kind, shader_name.c_str(), options);

  if (module.GetCompilationStatus() != shaderc_compilation_status_success) {
    std::cerr << module.GetErrorMessage();
    return std::vector<uint32_t>();
  }

  return {module.cbegin(), module.cend()};
}

int main(int argc, char** argv) {

  std::vector<uint32_t> vertex_spirv;
  VkShaderModule vertex_module;

  vertex_spirv = compile_shader(vertex_name,
                                shaderc_glsl_vertex_shader,
                                vertex_shader,
                                false);

  return EXIT_SUCCESS;
}

The shader code comes from the Vulkan tutorial (here), and the compile_shader function comes from the Google's shaderc library's example (here).

The compilation command: /usr/bin/c++ -I/media/PhantomData/Documents/Prog/carpCAD/inc -g -std=c++17 -O2 -MD -MT main.cpp.o -MF main.cpp.o.d -o main.cpp.o -c main.cpp The linking command: /usr/bin/c++ -g main.cpp.o -o foo -lshaderc -lMachineIndependent -lGenericCodeGen -lOGLCompiler -lOSDependent

When linking with -lshaderc only, I had lot of undefined reference to 'glslang::..., like in this ticket, and I found the missing libs to link with in this ticket.

When running it with Valgrind:

==28274== Command: ./foo
==28274==
==28274== Stack overflow in thread #1: can't grow stack to 0x1ffe801000
==28274==
==28274== Process terminating with default action of signal 11 (SIGSEGV)
==28274==  Access not within mapped region at address 0x1FFE801FE8
==28274== Stack overflow in thread #1: can't grow stack to 0x1ffe801000
==28274==    at 0x49392F3: ??? (in /usr/lib/x86_64-linux-gnu/libshaderc.so.1)
==28274==  If you believe this happened as a result of a stack
==28274==  overflow in your program's main thread (unlikely but
==28274==  possible), you can try to increase the size of the
==28274==  main thread stack using the --main-stacksize= flag.
==28274==  The main thread stack size used in this run was 8388608.

And the backtrace given by Gdb:

#0  0x00007ffff7ac92f3 in (anonymous namespace)::TGlslangToSpvTraverser::convertGlslangToSpvType(glslang::TType const&, glslang::TLayoutPacking, glslang::TQualifier const&, bool, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#1  0x00007ffff7acb71a in (anonymous namespace)::TGlslangToSpvTraverser::createSpvConstantFromConstUnionArray(glslang::TType const&, glslang::TConstUnionArray const&, int&, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#2  0x00007ffff7acba11 in (anonymous namespace)::TGlslangToSpvTraverser::createSpvConstantFromConstUnionArray(glslang::TType const&, glslang::TConstUnionArray const&, int&, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#3  0x00007ffff7acba11 in (anonymous namespace)::TGlslangToSpvTraverser::createSpvConstantFromConstUnionArray(glslang::TType const&, glslang::TConstUnionArray const&, int&, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
.....
#24917 0x00007ffff7acba11 in (anonymous namespace)::TGlslangToSpvTraverser::createSpvConstantFromConstUnionArray(glslang::TType const&, glslang::TConstUnionArray const&, int&, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#24918 0x00007ffff7acba11 in (anonymous namespace)::TGlslangToSpvTraverser::createSpvConstantFromConstUnionArray(glslang::TType const&, glslang::TConstUnionArray const&, int&, bool) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#24919 0x00007ffff7acc6a1 in (anonymous namespace)::TGlslangToSpvTraverser::visitConstantUnion(glslang::TIntermConstantUnion*) () from /lib/x86_64-linux-gnu/libshaderc.so.1
#24920 0x00007ffff7acf948 in (anonymous namespace)::TGlslangToSpvTraverser::visitBinary(glslang::TVisit, glslang::TIntermBinary*) () from /lib/x86_64-linux-gnu/libshaderc.so.1
#24921 0x000055555566de2a in glslang::TIntermBinary::traverse(glslang::TIntermTraverser*) ()
#24922 0x00007ffff7ae19f9 in (anonymous namespace)::TGlslangToSpvTraverser::visitAggregate(glslang::TVisit, glslang::TIntermAggregate*) () from /lib/x86_64-linux-gnu/libshaderc.so.1
#24923 0x000055555566e6fa in glslang::TIntermAggregate::traverse(glslang::TIntermTraverser*) ()
#24924 0x00007ffff7ae19f9 in (anonymous namespace)::TGlslangToSpvTraverser::visitAggregate(glslang::TVisit, glslang::TIntermAggregate*) () from /lib/x86_64-linux-gnu/libshaderc.so.1
#24925 0x000055555566e6fa in glslang::TIntermAggregate::traverse(glslang::TIntermTraverser*) ()
#24926 0x00007ffff7ae778e in glslang::GlslangToSpv(glslang::TIntermediate const&, std::vector<unsigned int, std::allocator<unsigned int> >&, spv::SpvBuildLogger*, glslang::SpvOptions*) ()
  from /lib/x86_64-linux-gnu/libshaderc.so.1
#24927 0x00007ffff7ae8e36 in glslang::GlslangToSpv(glslang::TIntermediate const&, std::vector<unsigned int, std::allocator<unsigned int> >&, glslang::SpvOptions*) () from /lib/x86_64-linux-gnu/libshaderc.so.1
#24928 0x00007ffff7ab59a6 in shaderc_util::Compiler::Compile(shaderc_util::string_piece const&, EShLanguage, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, std::function<EShLanguage (std::basic_ostream<char, std::char_traits<char> >*, shaderc_util::string_piece const&)> const&, shaderc_util::CountingIncluder&, shaderc_util::Compiler::OutputType, std::basic_ostream<char, std::char_traits<char> >*, unsigned long*, unsigned long*) const (this=this@entry=0x5555557c7378,
  input_source_string=..., forced_shader_stage=forced_shader_stage@entry=EShLangVertex,
  error_tag="vertex_shader", entry_point_name=entry_point_name@entry=0x55555571faaf "main",
  stage_callback=..., includer=..., output_type=<optimized out>, error_stream=<optimized out>,
  total_warnings=<optimized out>, total_errors=<optimized out>)
  at /usr/src/shaderc-2023.2-1/libshaderc_util/src/compiler.cc:328
#24929 0x00007ffff7ab857f in (anonymous namespace)::CompileToSpecifiedOutputType (
  compiler=<optimized out>, source_text=<optimized out>, source_text_size=<optimized out>,
  shader_kind=<optimized out>, input_file_name=<optimized out>, entry_point_name=<optimized out>,
  additional_options=<optimized out>, output_type=<optimized out>)
  at /usr/src/shaderc-2023.2-1/libshaderc/src/shaderc.cc:619
#24930 0x00007ffff7ab8d93 in shaderc_compile_into_spv (compiler=<optimized out>,
  source_text=<optimized out>, source_text_size=<optimized out>, shader_kind=<optimized out>,
  input_file_name=<optimized out>, entry_point_name=<optimized out>, additional_options=0x5555557c7370)
  at /usr/src/shaderc-2023.2-1/libshaderc/src/shaderc.cc:665
#24931 0x0000555555573958 in shaderc::Compiler::CompileGlslToSpv (options=...,
  entry_point_name=0x55555571faaf "main", input_file_name=<optimized out>,
  shader_kind=shaderc_vertex_shader, source_text_size=<optimized out>, source_text=<optimized out>,
  this=<optimized out>) at /usr/include/shaderc/shaderc.hpp:415
#24932 shaderc::Compiler::CompileGlslToSpv (options=..., input_file_name=<optimized out>,
  shader_kind=shaderc_vertex_shader, source_text_size=<optimized out>, source_text=<optimized out>,
  this=<optimized out>) at /usr/include/shaderc/shaderc.hpp:431
#24933 shaderc::Compiler::CompileGlslToSpv (this=<synthetic pointer>,
  source_text="\n#version 450\n\nvec2 positions[3] = vec2[](\n  vec2(0.0, -0.5),\n  vec2(0.5, 0.5),\n  vec2(-0.5, 0.5)\n);\n\nvec3 colors[3] = vec3[](\n  vec3(1.0, 0.0, 0.0),\n  vec3(0.0, 1.0, 0.0),\n  vec3(0.0, 0.0, 1.0)\n);\nla"..., options=<synthetic pointer>..., input_file_name=<optimized out>,
  shader_kind=shaderc_vertex_shader) at /usr/include/shaderc/shaderc.hpp:456
#24934 compile_shader (shader_name="vertex_shader", kind=kind@entry=shaderc_vertex_shader,
  source="\n#version 450\n\nvec2 positions[3] = vec2[](\n  vec2(0.0, -0.5),\n  vec2(0.5, 0.5),\n  vec2(-0.5, 0.5)\n);\n\nvec3 colors[3] = vec3[](\n  vec3(1.0, 0.0, 0.0),\n  vec3(0.0, 1.0, 0.0),\n  vec3(0.0, 0.0, 1.0)\n);\nla"..., optimize=optimize@entry=false) at main.cpp:52
#24935 0x0000555555572f83 in main (argc=<optimized out>, argv=<optimized out>) at main.cpp:67

I first thought that it could be because I declared the shader as a raw string, so I tried to declare is as a "normal" string with lot of \n but it changed nothing!

Am I missing something, or doing it wrong?

Thanks

Bardoux commented 5 months ago

I tried to reproduce it on the library compiled from the sources instead of the one from my packet manager. And, from sources on sha1 57d86ab7, I'm not able to hit this bug.

I guess this ticket may be closed.