Feature request: glslang compilation support

[Jkmcameron]

Vulkan support is already built into meson, if you install the LunarG Vulkan SDK anywhere on your PC then an environment variable is saved pointing to the SDK. dependency('vulkan') loads the vulkan libraries from that environment variable.

This brings me to my point, in modern graphics applications, dare I say in Vulkan applications in general, shaders are a necessity. Vulkan requires shaders, and though compilation at runtime is possible it is preferred and recommended that they be compiled beforehand.

The same Vulkan SDK that meson already locates when you call dependency('vulkan') also ships with a program called glslangvalidator that is used to compile shaders of various formats into the compiled Spir-V shaders that vulkan requires.

In my current project I've got this in my build file:

run_command('cmd', '/C', '"md "' + join_paths(meson.current_build_dir(), 'shaders') + '""')

foreach shader : shaders
    run_command('glslangvalidator', '-V', join_paths(meson.current_source_dir(), 'src', 'main', 'resource', 'shaders', shader) + '.glsl', '-o', join_paths(meson.current_build_dir(), 'shaders', shader + '.spv'))
endforeach

This works well enough (with a few caveats that I'm trying to solve) but I think it'd be a nice addition if this functionality would be built into meson, to be more inline with how C++ sources are defined in an array of strings and then compiled with a call to executable(), library(), etc.

Though I don't use Qt myself I see that meson also natively supports it, this to me shows that the team behind meson is open to adding features that assist developers that aren't limited to pure C/C++ compilation

[dcbaker]

(My day job is a working on mesa, the opensource OpenGL/Vulkan implementation), and I've been meaning to add a vulkan/opengl/spriv module to meson for this. In the mean time what you really want is not that, you want custom_targets.

glsllang = find_program('glslangvalidator')
shaders = ...  # full path with .glsl extension (or from subdir with files() extension)
shader_targets = []
foreach s : shaders
  shaders += custom_target(
    'shader @0@'.format(s),
    command : [glsllang, '-V', '@INPUT@', '-o',  '@OUTPUT@'],
    input : s, 
    output : '@BASENAME@.spv'.
  )
endforeach

You want custom_targets because then if the input glsl changes the output spir-v will be updated as well. It also means that you can pass these targets to other targets as inputs to create proper dependencies in the backend (ninja, msbuild), and this is run at build time instead of configure time.

[Jkmcameron]

You want custom_targets because then if the input glsl changes the output spir-v will be updated as well.

Thanks for that, Just about an hour ago I began wondering about this

[jpakkane]

The important thing is to find what functionality people actually have. For example should that validation thingy be a test? Should it be checked during compilation? Something else?

[Jkmcameron]

The important thing is to find what functionality people actually have. For example should that validation thingy be a test? Should it be checked during compilation? Something else?

it was just named poorly, it's closer to gcc or a compiler of sorts, the term validator should never have been tagged on the end. You write your shaders in GLSL, HLSL, or whatever shader language you would have been using prior to the introduction of Vulkan, and then compile those shaders to Spir-V, which is read by Vulkan. That's the gist of it

[Jkmcameron]

 command : [glsllang, '-V', '@INPUT@', '@OUTPUT@'],

Eh, I haven't tried this yet and I'm not sure of the exact usage of custom_target() but I believe -o belongs between @INPUT@ and @OUTPUT@, correct me if I'm wrong

[dcbaker]

err, yeah, you do need -o.

I edited the example in case someone in the future looks at this.

[dcbaker]

The important thing is to find what functionality people actually have. For example should that validation thingy be a test? Should it be checked during compilation? Something else?

LunarG wrote a bunch of spir-v helpers for cmake that are officially supported. I figured I'd look through the functionality they have as a starting point when I get to this.

[LDAP]

Using a meson generator you can already get pretty close:

system_glslc = find_program('glslangValidator')
glslc_path = system_glslc.full_path()

spv_compile_script = [meson.project_source_root() + '/scripts/compile_shader.py', '--glslc_path', glslc_path, '@EXTRA_ARGS@']
glslc_args = ['--target-env', 'vulkan1.3']
glslc_args += ['-I' + meson.project_source_root()]
glslc_args += ['-I' + join_paths(project_root, 'src')]
glslc_args += ['-I' + join_paths(project_root, 'include')]
glslc_args += ['--depfile', '@DEPFILE@']

py = import('python').find_installation('python3') 
shader_generator = generator(
    py,
    output: ['@PLAINNAME@.spv.h', '@PLAINNAME@.spv.c'],
    depfile: '@PLAINNAME@.d',
    arguments: spv_compile_script + ['@INPUT@', '@OUTPUT0@', '@OUTPUT1@'] + glslc_args,
)

The compile_shader.py script can be found here. It outputs a .h and a .c file containing the compiled shader.

Shaders can then be compiled like so:

src_files += shader_generator.process(
  'shader0.comp', 
  'shader1.comp', 
  extra_args: ['--prefix', 'my_prefix'],
  preserve_path_from: meson.source_root(),
)

extra_args and preserve_path_from are optional an can be used to prevent conflicts.