Compilation succeed with CLI but unexpected INTCONSTANT with C API

[tiawl]

Hi,

When using the C API with the following piece of code:

#include <cstring>
#include <iostream>

#include <shaderc/shaderc.h>

shaderc_include_result* resolve_include (void* user_data,
  const char* requested_source, int type,
  const char* requesting_source, size_t include_depth)
{
  const char source_name[] = "add.frag";
  const char content[] =
    "float add (float a, float b)\n"
    "{\n"
    "  return a + b;\n"
    "}\n";
  shaderc_include_result* result = new shaderc_include_result;
  result->source_name = source_name;
  result->source_name_length = std::strlen (source_name);
  result->content = content;
  result->content_length = std::strlen (content);
  result->user_data = nullptr;

  return result;
}

void release_include (void* user_data, shaderc_include_result* include_result)
{
  delete include_result;
}

int main ()
{
  const char source[] =
    "#version 450\n"
    "#include \"add.frag\"\n"
    "layout (location = 0) out vec4 o;\n"
    "void main ()\n"
    "{\n"
    "  o = vec4 (add (0.25, 0.4));\n"
    "}\n";

  shaderc_compiler_t compiler = shaderc_compiler_initialize ();
  shaderc_compile_options_t options = shaderc_compile_options_initialize ();

  shaderc_compile_options_set_include_callbacks (options,
    resolve_include, release_include, nullptr);

  shaderc_compilation_result_t result = shaderc_compile_into_spv (
      compiler, source, std::strlen (source), shaderc_glsl_fragment_shader,
      "test.frag", "main", options);

  shaderc_compilation_status status =
    shaderc_result_get_compilation_status (result);
  std::cout << "Result code " << int (status) << std::endl;

  if (status != shaderc_compilation_status_success)
  {
    std::cout << "error: " << shaderc_result_get_error_message (result) << std::endl;
  }

  shaderc_result_release (result);
  shaderc_compile_options_release (options);
  shaderc_compiler_release (compiler);
  return 0;
}

I have this strange output:

Result code 2
error: TV:1: error: '' :  syntax error, unexpected INTCONSTANT
       ^^
       Those characters change after each run

I cannot replicate the problem with the glslc command line tool. The version of libshaderc & glslc command line tool I am using, come from the v2024.0 tagged source:

$ glslc --version
shaderc v2023.8 v2024.0-1-g9a658e2
spirv-tools v2024.1 v2022.4-451-g04896c46
glslang 11.1.0-937-gd73712b8

Target: SPIR-V 1.0

Thank you

[tiawl]

The output is different with a build from the last commit:

Result code 2
error: test.frag:0: error: '#line' : unexpected tokens following directive
test.frag:0: error: 'string' : End of line in string
test.frag:1: error: '' :  syntax error, unexpected INTCONSTANT

[tiawl]

I am investigating a little bit. It seems to come from this part of the code into Compiler::Compile ():

  bool success = shader.parse(&limits_, default_version_, default_profile_,
                              force_version_profile_, kNotForwardCompatible,
                              rules, includer)

So it is a glslang::TShader::parse () issue. Because the only difference between the 2 commits was a dependencies update this is why the output changed. But it does not explain why glslc works and but not the shaderc C API.

[tiawl]

Here the value of the variables before going into the parse function from the libshaderc and the glslc usage (I used diff between the 2 outputs to be sure):

shader_strings = "#version 450
#include "add.frag"
layout (location = 0) out vec4 o;
void main ()
{
  o = vec4 (add (0.25, 0.4));
}
"
shader_lengths = 114
string_names = "test.frag"
preamble = "#extension GL_GOOGLE_include_directive : enable
"
entry_point_name = "main"
auto_bind_uniforms_ = 0
auto_map_locations_ = 0
hlsl_iomap_ = 0
used_shader_stage = 4
target_client_info.client = 1
target_client_info.client_version = 4194304
target_client_info.target_language = 1
target_client_info.target_language_version = 65536
invert_y_enabled_ = 0
nan_clamp_ = 0
limits_.maxLights = 8
limits_.maxClipPlanes = 6
limits_.maxTextureUnits = 2
limits_.maxTextureCoords = 8
limits_.maxVertexAttribs = 16
limits_.maxVertexUniformComponents = 4096
limits_.maxVaryingFloats = 60
limits_.maxVertexTextureImageUnits = 16
limits_.maxCombinedTextureImageUnits = 80
limits_.maxTextureImageUnits = 16
limits_.maxFragmentUniformComponents = 1024
limits_.maxDrawBuffers = 8
limits_.maxVertexUniformVectors = 256
limits_.maxVaryingVectors = 15
limits_.maxFragmentUniformVectors = 256
limits_.maxVertexOutputVectors = 16
limits_.maxFragmentInputVectors = 15
limits_.minProgramTexelOffset = -8
limits_.maxProgramTexelOffset = 7
limits_.maxClipDistances = 8
limits_.maxComputeWorkGroupCountX = 65535
limits_.maxComputeWorkGroupCountY = 65535
limits_.maxComputeWorkGroupCountZ = 65535
limits_.maxComputeWorkGroupSizeX = 1024
limits_.maxComputeWorkGroupSizeY = 1024
limits_.maxComputeWorkGroupSizeZ = 64
limits_.maxComputeUniformComponents = 512
limits_.maxComputeTextureImageUnits = 16
limits_.maxComputeImageUniforms = 8
limits_.maxComputeAtomicCounters = 8
limits_.maxComputeAtomicCounterBuffers = 1
limits_.maxVaryingComponents = 60
limits_.maxVertexOutputComponents = 64
limits_.maxGeometryInputComponents = 64
limits_.maxGeometryOutputComponents = 128
limits_.maxFragmentInputComponents = 128
limits_.maxImageUnits = 8
limits_.maxCombinedImageUnitsAndFragmentOutputs = 8
limits_.maxCombinedShaderOutputResources = 8
limits_.maxImageSamples = 0
limits_.maxVertexImageUniforms = 0
limits_.maxTessControlImageUniforms = 0
limits_.maxTessEvaluationImageUniforms = 0
limits_.maxGeometryImageUniforms = 0
limits_.maxFragmentImageUniforms = 8
limits_.maxCombinedImageUniforms = 8
limits_.maxGeometryTextureImageUnits = 16
limits_.maxGeometryOutputVertices = 256
limits_.maxGeometryTotalOutputComponents = 1024
limits_.maxGeometryUniformComponents = 512
limits_.maxGeometryVaryingComponents = 60
limits_.maxTessControlInputComponents = 128
limits_.maxTessControlOutputComponents = 128
limits_.maxTessControlTextureImageUnits = 16
limits_.maxTessControlUniformComponents = 1024
limits_.maxTessControlTotalOutputComponents = 4096
limits_.maxTessEvaluationInputComponents = 128
limits_.maxTessEvaluationOutputComponents = 128
limits_.maxTessEvaluationTextureImageUnits = 16
limits_.maxTessEvaluationUniformComponents = 1024
limits_.maxTessPatchComponents = 120
limits_.maxPatchVertices = 32
limits_.maxTessGenLevel = 64
limits_.maxViewports = 16
limits_.maxVertexAtomicCounters = 0
limits_.maxTessControlAtomicCounters = 0
limits_.maxTessEvaluationAtomicCounters = 0
limits_.maxGeometryAtomicCounters = 0
limits_.maxFragmentAtomicCounters = 8
limits_.maxCombinedAtomicCounters = 8
limits_.maxAtomicCounterBindings = 1
limits_.maxVertexAtomicCounterBuffers = 0
limits_.maxTessControlAtomicCounterBuffers = 0
limits_.maxTessEvaluationAtomicCounterBuffers = 0
limits_.maxGeometryAtomicCounterBuffers = 0
limits_.maxFragmentAtomicCounterBuffers = 0
limits_.maxCombinedAtomicCounterBuffers = 1
limits_.maxAtomicCounterBufferSize = 32
limits_.maxTransformFeedbackBuffers = 4
limits_.maxTransformFeedbackInterleavedComponents = 64
limits_.maxCullDistances = 8
limits_.maxCombinedClipAndCullDistances = 8
limits_.maxSamples = 4
limits_.maxMeshOutputVerticesNV = 256
limits_.maxMeshOutputPrimitivesNV = 512
limits_.maxMeshWorkGroupSizeX_NV = 32
limits_.maxMeshWorkGroupSizeY_NV = 1
limits_.maxMeshWorkGroupSizeZ_NV = 1
limits_.maxTaskWorkGroupSizeX_NV = 32
limits_.maxTaskWorkGroupSizeY_NV = 1
limits_.maxTaskWorkGroupSizeZ_NV = 1
limits_.maxMeshViewCountNV = 4
limits_.maxMeshOutputVerticesEXT = 256
limits_.maxMeshOutputPrimitivesEXT = 256
limits_.maxMeshWorkGroupSizeX_EXT = 128
limits_.maxMeshWorkGroupSizeY_EXT = 128
limits_.maxMeshWorkGroupSizeZ_EXT = 128
limits_.maxTaskWorkGroupSizeX_EXT = 128
limits_.maxTaskWorkGroupSizeY_EXT = 128
limits_.maxTaskWorkGroupSizeZ_EXT = 128
limits_.maxMeshViewCountEXT = 4
limits_.maxDualSourceDrawBuffersEXT = 1
limits_.limits.nonInductiveForLoops = 1
limits_.limits.whileLoops = 1
limits_.limits.doWhileLoops = 1
limits_.limits.generalUniformIndexing = 1
limits_.limits.generalAttributeMatrixVectorIndexing = 1
limits_.limits.generalVaryingIndexing = 1
limits_.limits.generalSamplerIndexing = 1
limits_.limits.generalVariableIndexing = 1
limits_.limits.generalConstantMatrixVectorIndexing = 1
default_version_ = 110
default_profile_ = 1
force_version_profile_ = 0
kNotForwardCompatible = 0
rules = 152

I am going to investigate into the glslang::TShader::parse () function.

[tiawl]

A first fix into my code: writting this:

shaderc_compilation_result_t result = shaderc_compile_into_spv (
      compiler, source, std::strlen (source), shaderc_glsl_default_fragment_shader,
      "test.frag", "main", options);

instead of this:

shaderc_compilation_result_t result = shaderc_compile_into_spv (
      compiler, source, std::strlen (source), shaderc_glsl_fragment_shader,
      "test.frag", "main", options);

allowed to trigger this conditionnal into Compiler::Compile ():

if (output_type == OutputType::PreprocessedText ||
    used_shader_stage == EShLangCount) {

But it is not enough, now I have this output:

Result code 2
error: test.frag:1: error: '' :  syntax error, unexpected INTCONSTANT

[dj2]

Out of curiosity , the two strings:

  const char source_name[] = "add.frag";
  const char content[] =
    "float add (float a, float b)\n"
    "{\n"
    "  return a + b;\n"
    "}\n";

Are declared inside the resolve_include function. You then return that address in the shaderc_include_result. When those get used, the function has returned and that stack frame is gone. If you move those two shaders out of the function into the global scope does that fix the issue?

That would also explain why the "name" of the file is randomly changing to things like TV. It's whatever happens to be in that memory now.

[tiawl]

Wow, what a ridiculous mistake it was. I was diving into the yyparse () glslang hell function when I saw your life-saving message. You saved my afternoon. Now everything goes well, thank you !