Provide a method for parsing Godot-style `.glsl` dynamically in extensions

[saki7]

Describe the project you are working on

A 2D Visual Novel game, features lots of advanced shading techniques such as anime-style composition

Describe the problem or limitation you are having in your project

.glsl files which contain Godot-style preprocessor tokens (such as #[compute] and #include) can not be compiled dynamically due to the lack of public API.

Whenever we pass Godot-style .glsl files to RenderingDevice::shader_compile_spirv_from_source(...), it will almost certainly result in compile error because the shader compiler will not understand Godot-specific preprocessor tokens.

While it can be statically imported to the Editor, one might alternatively generate a compute shader's source code and compile it on-the-fly. This sort of approach is quite convenient when the algorithm can be optimized (or can be switched to another one) depending on runtime conditions. If we can't compile compute shaders at runtime, those sort of optimizations can not be implemented.

Note that ResourceLoader::load(...) will never re-compile the shaders even when CACHE_MODE_IGNORE is specified, since the import was already done by ResourceImporterShaderFile and the .res file remains same, thus effectively ignoring the changes in original .glsl file. In other words, there exists no method to trigger re-compilation of .glsl file from GDExtension, even if it had been initially imported via Editor.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Currently the only official implementation for parsing #[compute] thingy resides in RDShaderFile::parse_versions_from_text(...):

https://github.com/godotengine/godot/blob/ccaebcd5e04aba9c7e3c9d819e4d10a40dd9679c/servers/rendering/rendering_device_binds.cpp#L60-L70

I suggest that this function must be exposed in RDShaderFile::_bind_methods, so that GDExtension users can call it.

We also need to improve the function interface, since the parameter like OpenIncludeFunction p_include_func and void *p_include_func_userdata looks nasty, and the specification is currently hidden.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Godot Editor already uses the function RDShaderFile::parse_versions_from_text(...) to import .glsl files. If we expose that function for GDExtension then we will be able to parse exact same Godot-style .glsl syntax without the need to import them into Editor.

If this enhancement will not be used often, can it be worked around with a few lines of script?

It is impossible to emulate the Godot-style parser, since the grammar is only expressed inside the engine core, and it can not be called directly from GDExtension.

Is there a reason why this should be core and not an add-on in the asset library?

It's a missing feature of core.

[saki7]

moved from godotengine/godot#76022

[saki7]

I think we need to discuss these points

• What is the appropriate name/interface for RDShaderFile::parse_versions_from_text(...)?
• Since Godot-style .glsl syntax is not compatible with GLSL in the first place, should we change the extension to something else, for instance .gdglsl?

[npip99]

If we could allow a .gdglsl extension that would be great. Right now there's no good way to have an IDE syntax highlight and lint a Godot-style GLSL. A GLSL linter complains constantly about the #[compute] directive. But, if I have setup a custom syntax that supports Godot-style GLSL, it invariably affects standard GLSL as well (For other OpenGL projects that I use my IDE for). I'd rather not use a Godot-style GLSL linter+syntax highlighter on GLSL files that have nothing to do with Godot.

• Does it seem unreasonable to just use .gdshader here? If .gdshader starts with #[compute], then the official GDShader syntax for that drops into GLSL-style syntax. Similar to how the IDE can recognize Assembly in C, when you use __asm__. If this isn't reasonable, .gdglsl would still resolve the issue.

[Calinou]

A GLSL linter complains constantly about the #[compute] directive.

That sounds like a bug with the linter. It should work with that directive, as anything that starts with # is valid GLSL (clang-format works fine with it). It's a preprocessor directive, and these are not 100% standardized across programs.

(Note: I explicitly mean GLSL, not Godot shader language here.)

[msemelman]

glslang (official linter) does not support #[compute]

glslang -S comp --glsl-version 450  shaders/tc_noise.glsl
shaders/tc_noise.glsl
ERROR: 0:1: '#' : invalid directive
ERROR: 0:1: '' : compilation terminated
ERROR: 2 compilation errors.  No code generated.