Comments stripped from GLSL to Metal transpile using spirv-cross

[alecazam]

We start with GLSL, run it through glslc to SPIRV, and then convert it to Metal via spirv-cross. The source code is in the SPIRV file when disassembled along with the SPIRV assembly code underneath. But I don't see any option in Spirv-Cross that preserves the comments from the original files. They're just all removed in the Metal code. Since this is the code that we're debugging and doing gpu capture on, and when spliced it's a large volume of code I didn't write, then I'd like for the comments to be preserved. Is there a spirv-cross option that I'm missing, or was this not included in this tool?

[alecazam]

I tried --emit-line-directives, but it seems to do nothing. Comments are just as important as the code in shaders, especially after the optimizer has simplified values like log2(2.8555) into constants and stuffed a bunch of temp vars into the original sources. I'm not even sure how a capture tool would remap transpiled Metal back to original headers/sources.

"SPIRV-Cross tries hard to emit readable and clean output from the SPIR-V. The goal is to emit GLSL or MSL that looks like it was written by a human and not awkward IR/assembly-like code."

[alecazam]

Seems like the parser has the OpLine directives and all of the source spliced into the spirv file. So it can lookup and see if there are one more lines of comment above those OpLine directives and copy them to a cache for the Metal/GLSL/HLSL transpile to copy into the output. Instead, it looks like only the GLSL tries to output these as line directives, and MSL/HLSL just ignore OpLine.

[HansKristian-Work]

Comments are not preserved in SPIR-V, so I don't see what SPIRV-Cross can possibly do here. OpSource would have the raw stuff, but attempting to splice in comments sounds like a hot mess.

[HansKristian-Work]

I would like to see a more concrete POC on how this could work, since I can't see how it would actually work in practice.

[alecazam]

First OpLine directives could be added by the compiler for comments, and changed to hold a range or at least a type for each comment, but that's not currently done or required for this. Also just pointing an OpLine to the start of a comment would identify that it is a comment when that data is looked up, but /* comments would need more parsing work without a range.

The OpLine is already the file/line for the code reference. If that references code with a comment above it, and a debug flag is set, then copy that line(s) into an array. Otherwise just push empty string, so that the lookup below can proceed.

And then when visiting the OpLine to produce the transpiled output, copy the array data out to the transpiled output. The source code is already embedded in the spv file, so it's not like any file operations are needed. Multiline and /* would be bonus points.

This will at least copy in function comments, and one liners above a complex piece of code. Probably won't work to copy trailing // comments on a line without actually tagging those with OpLine line/column directives as I suggested above.

 OpLine %1 119 0 <- this is the OpLine currently ignored in transpiles

And this is the code that it's referencing, that's already embedded

        // this is code with a comment above it
    highp vec3 v_worldPos = u_eyePos - v_eye; <- line 119, column 0

[alecazam]

Basically the currently embedded GLSL from glslc in the spv files is worthless otherwise if you transpile to HLSL or MSL. So then I'm stuck with compiling with a compiler that embeds my original shader source type. I have to use fxc or glslc to embed HLSL, or glslc to embed GLSL, or MSL which doesn't generate SPV at all yet and so isn't usable to embed source or embed gen spv.

So really the only thing of value in that embedded source is the commentary that can be transferred to the transpiled sources. Worst case is that more OpLine directive are added that reference in-line comments, but that's not a big deal and the directives are small. This is all for debug builds anyways.

Simple case above is O(N) time, so this adds little time to the transpile, and offers a lot of potential benefit.