Shader reflection API issues

[Lumengine]

Hi,

I don't know if it's the right place to ask questions but I will post it here and if it is not appropriate, I can move the discussion elsewhere. I am using slang reflection API to feed my engine but I have some troubles.

For some UTs, I have this shader in slang:

struct Stuff
{
    int a;
    float b;
};

struct Things
{
    Texture2D t;
    ParameterBlock<Stuff> stuff;
};

ParameterBlock<Stuff> gStuff;
Texture2D gTex;
ConstantBuffer<Things> gThings;
SamplerState s;

RWStructuredBuffer<float> result;

[shader("compute")]
[numthreads(16, 16, 1)]
void main(uint3 threadId: SV_DispatchThreadID)
{
    result[0] = (float)gStuff.a + gStuff.b + gTex.SampleLevel(s, float2(0, 0), 0).x + (float)gThings.stuff.a+ gThings.stuff.b + gThings.t.SampleLevel(s, float2(0, 0), 0).x;
}

This is just for unit testing so don't look at the stuff in main. That will compil into this : (with latest branch 2024.1.18

#version 450
layout(row_major) uniform;
layout(row_major) buffer;

#line 18 0
layout(std430, binding = 3) buffer StructuredBuffer_float_t_0 {
    float _data[];
} result_0;

#line 4
struct Stuff_std140_0
{
    int a_0;
    float b_0;
};

layout(binding = 0, set = 1)
layout(std140) uniform _S1
{
    int a_0;
    float b_0;
}gStuff_0;

#line 14
layout(binding = 0)
uniform texture2D gTex_0;

#line 16
layout(binding = 2)
uniform sampler s_0;

#line 16
layout(binding = 1)
uniform texture2D gThings_t_0;

#line 16
layout(binding = 0, set = 2)
layout(std140) uniform _S2
{
    int a_0;
    float b_0;
}gThings_stuff_0;
layout(local_size_x = 16, local_size_y = 16, local_size_z = 1) in;
void main()
{

#line 24
    const vec2 _S3 = vec2(0.0, 0.0);

#line 24
    result_0._data[0U] = float(gStuff_0.a_0) + gStuff_0.b_0 + (textureLod(sampler2D(gTex_0,s_0), (_S3), (0.0))).x + float(gThings_stuff_0.a_0) + gThings_stuff_0.b_0 + (textureLod(sampler2D(gThings_t_0,s_0), (_S3), (0.0))).x;
    return;
}

What I am trying to achieve is to get the correct binding and set for Vulkan for each variable. This shader is basded on an old comment I found in your repo when you were still using childSet in SimpleBindingOffset.

I am using this to compute the relative binding and set of variable

        /// Create an offset based on offset information in the given Slang `varLayout`
        SimpleBindingOffset(slang::VariableLayoutReflection* varLayout)
        {
            if (varLayout)
            {
                bindingSet = varLayout->getBindingSpace() + (Uint32)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE);
                binding = (Uint32)(varLayout->getOffset(SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT));
                pushConstantRange =
                    (uint32_t)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_PUSH_CONSTANT_BUFFER);
            }
        }

And I am adding recursively the offset with the parent. THe thing is that I get binding = 1 and set = 2 for gThings. But the gThings.t is in set 0 as the GLSL is showing (and binding 1). How am I supposed to get the right result ?

I read the documentation but I still have trouble understanding everything.

You can see part of my code here! I modified it a bit since last time but nothing fancy. https://github.com/shader-slang/slang/issues/3318

Thanks,

Baptiste

[csyonghe]

I think you should be fine by just summing up all getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE) from the var layouts on the path the to variable. getBindingSpace() shouldn't be needed if you are summing up things recursively in this manner.

[Lumengine]

Ok I deleted the getBindingSpace() from the code so now,

 if (varLayout)
 {
     bindingSet = (Uint32)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_SUB_ELEMENT_REGISTER_SPACE);
     binding = (Uint32)(varLayout->getOffset(SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT));
     pushConstantRange =
         (uint32_t)varLayout->getOffset(SLANG_PARAMETER_CATEGORY_PUSH_CONSTANT_BUFFER);
 }

I've got this code for each layout and summing up for the children recusively. But it doesn't help with the bug. gThings is still in set 2 and binding 1 but in the real code gThings_t_0 is in set 0 binding 1 and gThings_stuff_0 is in set 2 binding 0. I saw something with the category, maybe it comesfrom here ? For gThings the slang::ParameterCategory is Mixed and the category count is 2. Should I do moething from here ?

Thanks,

Baptiste

[Lumengine]

No idea ?

[csyonghe]

I will need to take a look at our reflection data to give you a better answer. You can try our reflection dumping tool to get a sense of the full reflection structure.

The tool is in tools/slang-reflection-test. You can run

slang-reflection-test a.slang

To see a full dump of the reflection info.

[Lumengine]

Ok thanks ! I'll try it.

[Lumengine]

And btw removing the getBindingSpace() makes some other shader reflection issues on my shaders. So for now, I'll let it until I figured out what's wrong. Thank you for your time :)

[Lumengine]

Sorry, I don't see an exe called slang-reflection-test. I see the slang-test.exe sio I tried something like this ./slang-test slang-reflection-test (I rapidly read the code, it seems this is the way to do ?) and it seems the output should be in the console but there is nothing as output. Am I doing something wrong ? Thanks, Baptiste

[csyonghe]

You are right, I apologize for my mistake. Currently slang-reflection-test is not compiled as a standalone exe but as a dll that is called by slang-test.exe.

You can run reflection test by compiling slang-reflection-test to exe, or just use slang-test.exe using the following steps:

1. Add the following comment line to your shader:

   //TEST:REFLECTION:-target spirv

   Note that there should be no space between // and TEST.

2. Put your file under <slang-repo-dir>/tests
3. Run slang-test from <slang-repo-dir> with argument: tests/your-shader-file.

[Lumengine]

This is the output !

result code = 0
standard error = {
}
standard output = {
{
    "parameters": [
        {
            "name": "gStuff",
            "binding": {"kind": "subElementRegisterSpace", "index": 1},
            "type": {
                "kind": "parameterBlock",
                "elementType": {
                    "kind": "struct",
                    "name": "Stuff",
                    "fields": [
                        {
                            "name": "a",
                            "type": {
                                "kind": "scalar",
                                "scalarType": "int32"
                            },
                            "binding": {"kind": "uniform", "offset": 0, "size": 4}
                        },
                        {
                            "name": "b",
                            "type": {
                                "kind": "scalar",
                                "scalarType": "float32"
                            },
                            "binding": {"kind": "uniform", "offset": 4, "size": 4}
                        }
                    ]
                },
                "containerVarLayout": {
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 0},
                        {"kind": "subElementRegisterSpace", "index": 0}
                    ]
                },
                "elementVarLayout": {
                    "type": {
                        "kind": "struct",
                        "name": "Stuff",
                        "fields": [
                            {
                                "name": "a",
                                "type": {
                                    "kind": "scalar",
                                    "scalarType": "int32"
                                },
                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                            },
                            {
                                "name": "b",
                                "type": {
                                    "kind": "scalar",
                                    "scalarType": "float32"
                                },
                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                            }
                        ]
                    },
                    "binding": {"kind": "uniform", "offset": 0, "size": 16}
                }
            }
        },
        {
            "name": "gTex",
            "binding": {"kind": "descriptorTableSlot", "index": 0},
            "type": {
                "kind": "resource",
                "baseShape": "texture2D"
            }
        },
        {
            "name": "gThings",
            "bindings": [
                {"kind": "descriptorTableSlot", "index": 1},
                {"kind": "subElementRegisterSpace", "index": 2}
            ],
            "type": {
                "kind": "constantBuffer",
                "elementType": {
                    "kind": "struct",
                    "name": "Things",
                    "fields": [
                        {
                            "name": "t",
                            "type": {
                                "kind": "resource",
                                "baseShape": "texture2D"
                            },
                            "binding": {"kind": "descriptorTableSlot", "index": 0}
                        },
                        {
                            "name": "stuff",
                            "type": {
                                "kind": "parameterBlock",
                                "elementType": {
                                    "kind": "struct",
                                    "name": "Stuff",
                                    "fields": [
                                        {
                                            "name": "a",
                                            "type": {
                                                "kind": "scalar",
                                                "scalarType": "int32"
                                            },
                                            "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                        },
                                        {
                                            "name": "b",
                                            "type": {
                                                "kind": "scalar",
                                                "scalarType": "float32"
                                            },
                                            "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                        }
                                    ]
                                },
                                "containerVarLayout": {
                                    "bindings": [
                                        {"kind": "descriptorTableSlot", "index": 0},
                                        {"kind": "subElementRegisterSpace", "index": 0}
                                    ]
                                },
                                "elementVarLayout": {
                                    "type": {
                                        "kind": "struct",
                                        "name": "Stuff",
                                        "fields": [
                                            {
                                                "name": "a",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "int32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                            },
                                            {
                                                "name": "b",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "float32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                            }
                                        ]
                                    },
                                    "binding": {"kind": "uniform", "offset": 0, "size": 16}
                                }
                            },
                            "binding": {"kind": "subElementRegisterSpace", "index": 0}
                        }
                    ]
                },
                "containerVarLayout": {

                },
                "elementVarLayout": {
                    "type": {
                        "kind": "struct",
                        "name": "Things",
                        "fields": [
                            {
                                "name": "t",
                                "type": {
                                    "kind": "resource",
                                    "baseShape": "texture2D"
                                },
                                "binding": {"kind": "descriptorTableSlot", "index": 0}
                            },
                            {
                                "name": "stuff",
                                "type": {
                                    "kind": "parameterBlock",
                                    "elementType": {
                                        "kind": "struct",
                                        "name": "Stuff",
                                        "fields": [
                                            {
                                                "name": "a",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "int32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                            },
                                            {
                                                "name": "b",
                                                "type": {
                                                    "kind": "scalar",
                                                    "scalarType": "float32"
                                                },
                                                "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                            }
                                        ]
                                    },
                                    "containerVarLayout": {
                                        "bindings": [
                                            {"kind": "descriptorTableSlot", "index": 0},
                                            {"kind": "subElementRegisterSpace", "index": 0}
                                        ]
                                    },
                                    "elementVarLayout": {
                                        "type": {
                                            "kind": "struct",
                                            "name": "Stuff",
                                            "fields": [
                                                {
                                                    "name": "a",
                                                    "type": {
                                                        "kind": "scalar",
                                                        "scalarType": "int32"
                                                    },
                                                    "binding": {"kind": "uniform", "offset": 0, "size": 4}
                                                },
                                                {
                                                    "name": "b",
                                                    "type": {
                                                        "kind": "scalar",
                                                        "scalarType": "float32"
                                                    },
                                                    "binding": {"kind": "uniform", "offset": 4, "size": 4}
                                                }
                                            ]
                                        },
                                        "binding": {"kind": "uniform", "offset": 0, "size": 16}
                                    }
                                },
                                "binding": {"kind": "subElementRegisterSpace", "index": 0}
                            }
                        ]
                    },
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 0},
                        {"kind": "subElementRegisterSpace", "index": 0}
                    ]
                }
            }
        },
        {
            "name": "s",
            "binding": {"kind": "descriptorTableSlot", "index": 2},
            "type": {
                "kind": "samplerState"
            }
        },
        {
            "name": "result",
            "binding": {"kind": "descriptorTableSlot", "index": 3},
            "type": {
                "kind": "resource",
                "baseShape": "structuredBuffer",
                "access": "readWrite",
                "resultType": {
                    "kind": "scalar",
                    "scalarType": "float32"
                }
            }
        }
    ],
    "entryPoints": [
        {
            "name": "main",
            "stage:": "compute",
            "parameters": [
                {
                    "name": "threadId",
                    "semanticName": "SV_DISPATCHTHREADID",
                    "type": {
                        "kind": "vector",
                        "elementCount": 3,
                        "elementType": {
                            "kind": "scalar",
                            "scalarType": "uint32"
                        }
                    }
                }
            ],
            "threadGroupSize": [16, 16, 1],
            "bindings": [
                {
                    "name": "gStuff",
                    "binding": {"kind": "subElementRegisterSpace", "index": 1}
                },
                {
                    "name": "gTex",
                    "binding": {"kind": "descriptorTableSlot", "index": 0}
                },
                {
                    "name": "gThings",
                    "bindings": [
                        {"kind": "descriptorTableSlot", "index": 1},
                        {"kind": "subElementRegisterSpace", "index": 2}
                    ]
                },
                {
                    "name": "s",
                    "binding": {"kind": "descriptorTableSlot", "index": 2}
                },
                {
                    "name": "result",
                    "binding": {"kind": "descriptorTableSlot", "index": 3}
                }
            ]
        }
    ]
}
}

[Lumengine]

So if I understand correctly gThings has a parent binding to index 1 and set 2 but gTHings.t has a descriptorTableSlot to 0 as well as gThings.gStuff. I know it's a weird case but I'm just creating UT and this one comes from one of your old repo.