search

baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
9.06k stars 1.35k forks source link

Can't compile shaders in Edit mode #3448

Closed Goshido closed 1 month ago

Goshido commented 1 month ago

Description

I'm using Vulkan + HLSL workflow. After editing shader and press compile DXC report compilation error:

Running "D:/Development/VulkanSDK/1.3.290.0/Bin/dxc.exe" -T ps_6_0 -E PS -spirv -fspv-target-env=vulkan1.1 -Fo C:/Users/Goshido/AppData/Local/Temp/shader_output C:/Users/Goshido/AppData/Local/Temp/shader_input
error: missing entry point definition

Process crashed with code -2147467259.
Output file is 0 bytes

I also noticed that previous RenderDoc v1.34 showed exactly the same compilation flag information:

-T {hlsl_stage2}_6_0 -E {entry_point} -spirv -fspv-target-env={vulkan_ver}  -E PS -T ps_6_8 -HV 2021 -spirv 
-fvk-use-dx-layout -fspv-reduce-load-size -fspv-target-env=vulkan1.1 -enable-16bit-types -WX -I tools\editor\hlsl
-I tools\editor\include -I app\src\main\hlsl -I app\src\main\cpp\include\pbr -Od -fspv-debug=vulkan-with-source
-Fo tools\editor\editor-assets\shaders\blit.ps.spv -Qembed_debug

Current RenderDoc v1.35 shows less information:

-T {hlsl_stage2}_6_0 -E {entry_point} -spirv -fspv-target-env={vulkan_ver}

Steps to reproduce

  1. Take the following shader code:

blit_program.ps:

#include "blit_program.inc"
#include "color_space.ps"
#include "srgb_program.inc"

// 1.0H / 2.4H
#define INVERSE_GAMMA           4.16667e-1F

[[vk::constant_id ( CONST_INVERSE_GAMMA )]]
float const                     g_inverseGamma = INVERSE_GAMMA;

[[vk::binding ( BIND_IMAGE, SET_BLIT )]]
Texture2D<float32_t4>           g_image:        register ( t0 );

[[vk::binding ( BIND_SAMPLER, SET_BLIT )]]
SamplerState                    g_sampler:      register ( s0 );

struct InputData
{
    [[vk::location ( ATT_SLOT_UV )]]
    noperspective float32_t2    _uv:            UV;
};

//----------------------------------------------------------------------------------------------------------------------

float32_t4 PS ( in InputData inputData ): SV_Target0
{
    float16_t3 const color = (float16_t3)g_image.SampleLevel ( g_sampler, inputData._uv, 0.0F ).xyz;
    return float32_t4 ( LinearToSRGB ( color, (float16_t)g_inverseGamma ), 1.0F );
}

color_space.ps:

#ifndef COLOR_SPACE_PS
#define COLOR_SPACE_PS

float32_t3 LinearToSRGB ( in float16_t3 ldr, in float16_t inverseGamma )
{
    // See https://entropymine.com/imageworsener/srgbformula/
    float16_t3 const a = ldr * 12.92H;
    float16_t3 const b = 1.055H * pow ( ldr, inverseGamma ) - 0.055H;
    return (float32_t3)lerp ( a, b, (float16_t3)( ldr > 3.1308e-3H ) );
}

#endif // COLOR_SPACE_PS

blit_program.inc:

#ifndef BLIT_PROGRAM_INC
#define BLIT_PROGRAM_INC

#define ATT_SLOT_UV     0

#define SET_BLIT        0
#define BIND_IMAGE      0
#define BIND_SAMPLER    1

#endif // BLIT_PROGRAM_INC

srgb_program.inc:

#ifndef SRGB_PROGRAM_INC
#define SRGB_PROGRAM_INC

#define CONST_INVERSE_GAMMA     0

#endif // SRGB_PROGRAM_INC
  1. Compile it with the following flags:
dxc.exe                                                     ^
    -HV 2021                                                ^
    -spirv                                                  ^
    -fvk-use-dx-layout                                      ^
    -fspv-reduce-load-size                                  ^
    -fspv-target-env=vulkan1.1                              ^
    -enable-16bit-types                                     ^
    -WX                                                     ^
    -I tools\editor\hlsl                                    ^
    -I tools\editor\include                                 ^
    -I app\src\main\hlsl                                    ^
    -I app\src\main\cpp\include\pbr                         ^
    -Od                                                     ^
    -fspv-debug=vulkan-with-source                          ^
    -E PS                                                   ^
    -T ps_6_8                                               ^
    -Fo tools\editor\editor-assets\shaders\blit.ps.spv      ^
    tools\editor\hlsl\blit.ps
  1. Use this program and made capture.
  2. Try to edit program blit_program.ps by multiplying color variable by 0.5H.
...

float32_t4 PS ( in InputData inputData ): SV_Target0
{
    float16_t3 const color = (float16_t3)g_image.SampleLevel ( g_sampler, inputData._uv, 0.0F ).xyz;
    return float32_t4 ( LinearToSRGB ( color * 0.5H, (float16_t)g_inverseGamma ), 1.0F ); <--------------------------
}
  1. Press apply changes.

Problem: Compilation fails:

Running "D:/Development/VulkanSDK/1.3.290.0/Bin/dxc.exe" -T ps_6_0 -E PS -spirv
-fspv-target-env=vulkan1.1 -Fo C:/Users/Goshido/AppData/Local/Temp/shader_output
C:/Users/Goshido/AppData/Local/Temp/shader_input
error: missing entry point definition

Process crashed with code -2147467259.
Output file is 0 bytes

Expected: Compilation success.

Environment

Additional information.

The video with problem:

https://github.com/user-attachments/assets/f0ef8f84-19f0-4230-9219-11175e0bf62c

baldurk commented 1 month ago

I can't reproduce this. I compiled the shaders as directed and the command line for the editing works in v1.34 and v1.35, though it fails due to the compiler bug you reported before in #3425 which is not a RenderDoc issue.

It would be easier if you could share your capture to reproduce the problem. Ideally one made in v1.34 which works and one made in v1.35 which breaks since you say it is a regression between versions.

Goshido commented 1 month ago

Sure. Here is captures:

From RenderDoc v1.34.

From RenderDoc v1.35.

baldurk commented 1 month ago

I get the failure to edit which seems to be caused by a broken shader - but both captures are equally broken for me so I'm presumably not seeing the same thing as you. Loading the v1.34 capture in either v1.34 or v1.35 is broken in exactly the same way as the v1.35 capture.

You're saying that this works in v1.34 with that capture, which draw exactly works in v1.34? does it also work if you load that working capture in v1.35?

Goshido commented 1 month ago

You're saying that this works in v1.34 with that capture, which draw exactly works in v1.34?

I double checked v1.34 behavior. It turns out that error exactly the same as in v1.35. So it's not a regression but the same bug.

does it also work if you load that working capture in v1.35?

No. It doesn't work. Compilation error happens as described in the issue.

It looks like the SPIR-V blobs from new DXC trigger compilation error in the RenderDoc. I mean new DXC fixes "shader source with define only includes" but breaks shader compilation. Maybe the problem is in SPIR-V blobs itself? In RenderDoc module which parses SPIR-V and extracts shader source information and compilation flags?

Here is blobs from old and new DXC: SPIR-V blobs.zip. Would you kindly run RenderDoc under debugger and compare outcome for that SPIR-V blobs?

baldurk commented 1 month ago

So you're saying that when you said it was working in v1.34 and only broke when you updated to v1.35, that's not the case, and the capture that you uploaded and said was working in v1.34, it never worked?

In that case I don't think there's a RenderDoc bug here, only the broken shaders. It sounds like you changed DXC versions and the new DXC version emits broken shaders.

For the future please be more careful when reporting bugs and don't provide incorrect information as it wastes time and causes confusion.

Goshido commented 1 month ago

@baldurk if I understood correctly it's not RenderDoc parsing issue. So I created the issue for DXC: #6939.