CMake rules create duplicate implicit targets on Windows

[vettoreldaniele]

There is a problem on how we generate CMake rules for shader targets, specifically on Windows.

We use MAIN_DEPENDENCY to specify the input shader file (<shader_name>.hlsl) when we create custom commands to compiler shaders with DXC (add_custom_command). We use MAIN_DEPENDENCY instead of DEPENDS because the former allows Visual Studio to avoid trying to compile the HLSL file with its built-in compiler (FXC). However, due to Visual Studio limitations, it is not possible to have multiple add_custom_command rules with the same MAIN_DEPENDENCY file. We do that when we compile the same shader for multiple stages. This does work in the sense that CMake generation succeeds, but it creates implicit multiple targets that cause Visual Studio to try to compile the same file multiple times in parallel causing sporadic build failures (in case the rules run in parallel).

To reproduce, try generating a VS solution and building a shader target with multiple stages.

I will add more specific information soon.

[vettoreldaniele]

Reproduce: 1) Generate a VS solution with CMake. To keep it simple, only enable Vulkan (-DPPX_D3D12=OFF -DPPX_VULKAN=ON). 2) Open the solution (BigWheels.sln). 3) Pick a shader target. I chose shader_skybox (under the shaders folder in the Solution View). 4) Right click the vertex shader target (shader_skybox_vs) and choose Rebuild (we're using Rebuild instead of Build to force it to clean and build from scratch so that it's reproducible). You will have the following output showing it compiled the vertex shader:

Rebuild started...
1>------ Rebuild All started: Project: vk_shader_skybox_vs, Configuration: Debug x64 ------
1>------ Compiling VS Shader [SPV_6_6] ------
1>[SPV_6_6] Compiling VS C:/src/bigwheels_fork/assets/basic/shaders/SkyBox.hlsl to C:/src/bigwheels_fork/build-windows/assets/basic/shaders/spv/SkyBox.vs.spv
1>Building Custom Rule C:/src/bigwheels_fork/assets/basic/shaders/CMakeLists.txt
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

5) Now right click the pixel shader target (shader_skybox_ps) and choose Rebuild. You will have the following output showing it compiled both the pixel shader target and the previous vertex shader target:

Rebuild started...
1>------ Rebuild All started: Project: vk_shader_skybox_ps, Configuration: Debug x64 ------
1>------ Compiling VS Shader [SPV_6_6] ------
1>[SPV_6_6] Compiling VS C:/src/bigwheels_fork/assets/basic/shaders/SkyBox.hlsl to C:/src/bigwheels_fork/build-windows/assets/basic/shaders/spv/SkyBox.vs.spv
1>------ Compiling PS Shader [SPV_6_6] ------
1>[SPV_6_6] Compiling PS C:/src/bigwheels_fork/assets/basic/shaders/SkyBox.hlsl to C:/src/bigwheels_fork/build-windows/assets/basic/shaders/spv/SkyBox.ps.spv
1>Building Custom Rule C:/src/bigwheels_fork/assets/basic/shaders/CMakeLists.txt
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

This is because Visual Studio can only attach CMake rules to input files and not to output files. Due to the fact that we attach multiple rules (vertex shader rule and pixel shader rule) to the same MAIN_DEPENDENCY (the shader file), this confuses Visual Studio.

So: 1) CMake generates the first rule (vertex shader) -> the shader file is being used as the MAIN_DEPENDENCY for the first time, all works correctly. 2) CMake then generates the second rule (pixel shader) -> the shader file was already used as MAIN_DEPENDENCY, so the generator attaches this additional rule AND the previous rule to the shader file, causing the problem.

[Keenuts]

Rules can generate 2 files right? If so maybe we could create a proxy rule, taking the original *.hlsl, copying it to *.hlsl.{vs,cs,ps} and then each rule would depend on that? But I suppose the issue is then if you build both VK and DX, as each would rely on the same *.hlsl.vs ?

[vettoreldaniele]

Rules can generate 2 files right? If so maybe we could create a proxy rule, taking the original *.hlsl, copying it to *.hlsl.{vs,cs,ps} and then each rule would depend on that? But I suppose the issue is then if you build both VK and DX, as each would rely on the same *.hlsl.vs ?

I thought about this, but then I wonder if the copy makes it so that the build system is not able to tell when a file was modified (and trigger a rebuild)? And yes it would need a copy for each API and each stage (so hlsl.dx.vs and hlsl.vk.vs, etc.)

[vettoreldaniele]

I've implemented a temporary CI workaround in https://github.com/google/bigwheels/pull/96 so that the shaders are built serially. Note that this problem will still appear in local builds, but it can be worked around by recompiling multiple times until there's no conflict.