Bug: In multiplies @ctype

[kochol]

When I have two ctypes at the beginning of my GLSL file. The compiler will ignore the first one but when I add an empty line at the beginning of my code It works again.

@ctype mat4 HMM_Mat4
@ctype vec4 HMM_Vec4

this generates float[16] for mat4 data types but when I add one empty line it works again.

Here is my shader code to reproduce the problem.

@ctype mat4 HMM_Mat4
@ctype vec4 HMM_Vec4

// shared code for all shaders
@block uniforms
uniform vs_grid_params {
    mat4 mvp;
    vec4 colors[4];
};
@end

// offscreen rendering shaders
@vs vs_grid
@include_block uniforms

in vec4 position;
out vec4 color;

void main() {
    gl_Position = mvp * vec4(position.xyz, 1);
    color = colors[int(round(position.w))];
}
@end

@fs fs_grid
in vec4 color;
out vec4 frag_color;

void main() {
    frag_color = color;
}
@end

@program grid vs_grid fs_grid

[floooh]

Hmm, I cannot reproduce this with the current sokol-shdc in https://github.com/floooh/sokol-tools-bin (and on macOS)

E.g. the generated struct for vs_grid_params looks like this:

#pragma pack(push,1)
SOKOL_SHDC_ALIGN(16) typedef struct vs_grid_params_t {
    HMM_Mat4 mvp;
    HMM_Vec4 colors[4];
} vs_grid_params_t;
#pragma pack(pop)

...compiled with the following command line: sokol-shdc -i bla.glsl -o bla.h -l glsl330

Can you provide more info?

• operating system
• the complete command line
• and the complete output file

Also, are you using the latest version from https://github.com/floooh/sokol-tools-bin?

PS: also can you attach the input file as-is instead of copy-pasting? Maybe it's some weird line ending problem which gets lost in the copy-paste...

[kochol]

Operation system is windows 11 pro

I use sokol_shader macro in my cmake files.

Here is the complete output file.

#pragma once
/*
    #version:5# (machine generated, don't edit!)

    Generated by sokol-shdc (https://github.com/floooh/sokol-tools)

    Cmdline: sokol-shdc --input C:/Users/Cyrus/Documents/GitHub/moco/src/moco/gfx/shaders/grid.glsl --output C:/Users/Cyrus/Documents/GitHub/fips-build/moco/win64-vstudio-debug/src/moco/grid.glsl.h --slang glsl330 --genver 5 --errfmt msvc --format sokol --bytecode

    Overview:

        Shader program 'grid':
            Get shader desc: grid_shader_desc(sg_query_backend());
            Vertex shader: vs_grid
                Attribute slots:
                    ATTR_vs_grid_position = 0
                Uniform block 'vs_grid_params':
                    C struct: vs_grid_params_t
                    Bind slot: SLOT_vs_grid_params = 0
            Fragment shader: fs_grid

    Shader descriptor structs:

        sg_shader grid = sg_make_shader(grid_shader_desc(sg_query_backend()));

    Vertex attribute locations for vertex shader 'vs_grid':

        sg_pipeline pip = sg_make_pipeline(&(sg_pipeline_desc){
            .layout = {
                .attrs = {
                    [ATTR_vs_grid_position] = { ... },
                },
            },
            ...});

    Image bind slots, use as index in sg_bindings.vs.images[] or .fs.images[]

    Sampler bind slots, use as index in sg_bindings.vs.sampler[] or .fs.samplers[]

    Bind slot and C-struct for uniform block 'vs_grid_params':

        vs_grid_params_t vs_grid_params = {
            .mvp = ...;
            .colors = ...;
        };
        sg_apply_uniforms(SG_SHADERSTAGE_[VS|FS], SLOT_vs_grid_params, &SG_RANGE(vs_grid_params));

*/
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
#include <stddef.h>
#if !defined(SOKOL_SHDC_ALIGN)
  #if defined(_MSC_VER)
    #define SOKOL_SHDC_ALIGN(a) __declspec(align(a))
  #else
    #define SOKOL_SHDC_ALIGN(a) __attribute__((aligned(a)))
  #endif
#endif
#define ATTR_vs_grid_position (0)
#define SLOT_vs_grid_params (0)
#pragma pack(push,1)
SOKOL_SHDC_ALIGN(16) typedef struct vs_grid_params_t {
    float mvp[16];
    HMM_Vec4 colors[4];
} vs_grid_params_t;
#pragma pack(pop)
/*
    #version 330

    uniform vec4 vs_grid_params[8];
    layout(location = 0) in vec4 position;
    out vec4 color;

    void main()
    {
        gl_Position = mat4(vs_grid_params[0], vs_grid_params[1], vs_grid_params[2], vs_grid_params[3]) * vec4(position.xyz, 1.0);
        color = vs_grid_params[int(round(position.w)) * 1 + 4];
    }

*/
static const char vs_grid_source_glsl330[306] = {
    0x23,0x76,0x65,0x72,0x73,0x69,0x6f,0x6e,0x20,0x33,0x33,0x30,0x0a,0x0a,0x75,0x6e,
    0x69,0x66,0x6f,0x72,0x6d,0x20,0x76,0x65,0x63,0x34,0x20,0x76,0x73,0x5f,0x67,0x72,
    0x69,0x64,0x5f,0x70,0x61,0x72,0x61,0x6d,0x73,0x5b,0x38,0x5d,0x3b,0x0a,0x6c,0x61,
    0x79,0x6f,0x75,0x74,0x28,0x6c,0x6f,0x63,0x61,0x74,0x69,0x6f,0x6e,0x20,0x3d,0x20,
    0x30,0x29,0x20,0x69,0x6e,0x20,0x76,0x65,0x63,0x34,0x20,0x70,0x6f,0x73,0x69,0x74,
    0x69,0x6f,0x6e,0x3b,0x0a,0x6f,0x75,0x74,0x20,0x76,0x65,0x63,0x34,0x20,0x63,0x6f,
    0x6c,0x6f,0x72,0x3b,0x0a,0x0a,0x76,0x6f,0x69,0x64,0x20,0x6d,0x61,0x69,0x6e,0x28,
    0x29,0x0a,0x7b,0x0a,0x20,0x20,0x20,0x20,0x67,0x6c,0x5f,0x50,0x6f,0x73,0x69,0x74,
    0x69,0x6f,0x6e,0x20,0x3d,0x20,0x6d,0x61,0x74,0x34,0x28,0x76,0x73,0x5f,0x67,0x72,
    0x69,0x64,0x5f,0x70,0x61,0x72,0x61,0x6d,0x73,0x5b,0x30,0x5d,0x2c,0x20,0x76,0x73,
    0x5f,0x67,0x72,0x69,0x64,0x5f,0x70,0x61,0x72,0x61,0x6d,0x73,0x5b,0x31,0x5d,0x2c,
    0x20,0x76,0x73,0x5f,0x67,0x72,0x69,0x64,0x5f,0x70,0x61,0x72,0x61,0x6d,0x73,0x5b,
    0x32,0x5d,0x2c,0x20,0x76,0x73,0x5f,0x67,0x72,0x69,0x64,0x5f,0x70,0x61,0x72,0x61,
    0x6d,0x73,0x5b,0x33,0x5d,0x29,0x20,0x2a,0x20,0x76,0x65,0x63,0x34,0x28,0x70,0x6f,
    0x73,0x69,0x74,0x69,0x6f,0x6e,0x2e,0x78,0x79,0x7a,0x2c,0x20,0x31,0x2e,0x30,0x29,
    0x3b,0x0a,0x20,0x20,0x20,0x20,0x63,0x6f,0x6c,0x6f,0x72,0x20,0x3d,0x20,0x76,0x73,
    0x5f,0x67,0x72,0x69,0x64,0x5f,0x70,0x61,0x72,0x61,0x6d,0x73,0x5b,0x69,0x6e,0x74,
    0x28,0x72,0x6f,0x75,0x6e,0x64,0x28,0x70,0x6f,0x73,0x69,0x74,0x69,0x6f,0x6e,0x2e,
    0x77,0x29,0x29,0x20,0x2a,0x20,0x31,0x20,0x2b,0x20,0x34,0x5d,0x3b,0x0a,0x7d,0x0a,
    0x0a,0x00,
};
/*
    #version 330

    layout(location = 0) out vec4 frag_color;
    in vec4 color;

    void main()
    {
        frag_color = color;
    }

*/
static const char fs_grid_source_glsl330[114] = {
    0x23,0x76,0x65,0x72,0x73,0x69,0x6f,0x6e,0x20,0x33,0x33,0x30,0x0a,0x0a,0x6c,0x61,
    0x79,0x6f,0x75,0x74,0x28,0x6c,0x6f,0x63,0x61,0x74,0x69,0x6f,0x6e,0x20,0x3d,0x20,
    0x30,0x29,0x20,0x6f,0x75,0x74,0x20,0x76,0x65,0x63,0x34,0x20,0x66,0x72,0x61,0x67,
    0x5f,0x63,0x6f,0x6c,0x6f,0x72,0x3b,0x0a,0x69,0x6e,0x20,0x76,0x65,0x63,0x34,0x20,
    0x63,0x6f,0x6c,0x6f,0x72,0x3b,0x0a,0x0a,0x76,0x6f,0x69,0x64,0x20,0x6d,0x61,0x69,
    0x6e,0x28,0x29,0x0a,0x7b,0x0a,0x20,0x20,0x20,0x20,0x66,0x72,0x61,0x67,0x5f,0x63,
    0x6f,0x6c,0x6f,0x72,0x20,0x3d,0x20,0x63,0x6f,0x6c,0x6f,0x72,0x3b,0x0a,0x7d,0x0a,
    0x0a,0x00,
};
#if !defined(SOKOL_GFX_INCLUDED)
  #error "Please include sokol_gfx.h before grid.glsl.h"
#endif
static inline const sg_shader_desc* grid_shader_desc(sg_backend backend) {
  if (backend == SG_BACKEND_GLCORE33) {
    static sg_shader_desc desc;
    static bool valid;
    if (!valid) {
      valid = true;
      desc.attrs[0].name = "position";
      desc.vs.source = vs_grid_source_glsl330;
      desc.vs.entry = "main";
      desc.vs.uniform_blocks[0].size = 128;
      desc.vs.uniform_blocks[0].layout = SG_UNIFORMLAYOUT_STD140;
      desc.vs.uniform_blocks[0].uniforms[0].name = "vs_grid_params";
      desc.vs.uniform_blocks[0].uniforms[0].type = SG_UNIFORMTYPE_FLOAT4;
      desc.vs.uniform_blocks[0].uniforms[0].array_count = 8;
      desc.fs.source = fs_grid_source_glsl330;
      desc.fs.entry = "main";
      desc.label = "grid_shader";
    }
    return &desc;
  }
  return 0;
}

[floooh]

Thanks for the additional info! I'll see if I can reproduce the problem on my Windows machine.

[floooh]

Hmm, still no luck reproducing this on my Windows machine, I copied your shader source into a text file, and ran your command line, and I'm getting the correct C struct:

SOKOL_SHDC_ALIGN(16) typedef struct vs_grid_params_t {
    HMM_Mat4 mvp;
    HMM_Vec4 colors[4];
} vs_grid_params_t;

I do have a theory though:

Windows editors (also Visual Studio) sometimes like to put some non-text bytes at the start of text files (so called 'BOM Markers'), even for UTF-8 encoded text files (https://en.wikipedia.org/wiki/Byte_order_mark). And if such bytes (e.g. EF BB BF for UTF-8) are present it might have the effect you are seeing. BOM markers for UTF-8 are not recommended by the UNICODE standard, but for some reason Microsoft didn't get the memo yet.

Can you take your original glsl file, compress it into a zip file, and upload the zip file here (just drag'n'drop into a comment should work).

[floooh]

PS: in Visual Studio the relevant encodings are called Unicode (UTF-8 with signature) (the bad version with BOM), and Unicode (UTF-8 without signature) (the correct version without BOM).

Those can be selected in the Save dialog by clicking on the down-arrow in the Save button and selecting "Save with Encoding".

[kochol]

I created my file with rider IDE.

grid.zip

[floooh]

Yep, theory confirmed, there's an EF BB BF BOM marker in front of the actual text:

00000000  ef bb bf 40 63 74 79 70  65 20 6d 61 74 34 20 48  |...@ctype mat4 H|
00000010  4d 4d 5f 4d 61 74 34 0d  0a 40 63 74 79 70 65 20  |MM_Mat4..@ctype |

Does Rider have an option to save UTF-8 without BOM? I could also put a specific hack to detect and skip the BOM in sokol-shdc, but since it's a (bad) Windows-ism, and the issue didn't pop up before I rather wouldn't.

[kochol]

I discovered that the Rider IDE has a button located on the bottom-right corner of the screen that allows you to add or remove the Byte Order Mark (BOM) from a file. I have attached an image of the button for your reference: After removing the BOM from the file, I can confirm that the bug has been fixed.