[SPIR-V] Vulkan problem with YCbCr immutable sampler when compiled using HLSL -> DXC, works in GLSL

[BattleAxeVR]

Hi there, I'm having a rather weird and probably very niche issue with my HLSL shaders to read Vulkan Video and was wondering if this is a known issue. I did a search here for ycbcr but didn't get any hits.

I posted the issue on the Vulkan Video github but I think the bug, if it is indeed a bug and not just an incorrect parameter for DXC.exe (or perhaps something missing in my HLSL code), is better posted here.

Anyway, here's the original:

https://github.com/nvpro-samples/vk_video_samples/issues/15

Basically, it's a very simple shader that works in GLSL -> glslLangvalidator -> Vulkan but not in HLSL -> DXC -> Vulkan.

There isn't anything different about using an immutable sampler vs a normal one in the shader bindings, or anything else I can tell, and the C/C++ Vulkan code driving them is exactly the same as well.

When I use GLSL it works, when I use HLSL it doesn't.

Since the code for sampling the texture is very straightforward, I don't see what I'm doing wrong in either shader code or C/C++ side (no validation errors either), so there is probably either a DXC parameter I'm missing, or some decoration syntax I'm missing, or perhaps simply that DXC.exe won't work with this use case.

Which is OK for now, I have a workaround (use GLSL for this one shader just to decode YUV 420 HEVC streams to RGB), but that doubles the memory use compared to sampling it directly in my main rendering. Actually, it triples it, since there is an extra full RGB copy of the original on top of the original. Anyway, I'd love to find out if anyone knows about this! Or has any suggestions for DXC compilation parameters I might try to make my HLSL output the same SPIR-V binary as GLSL does.

[BattleAxeVR]

Any updates on this?

[sudonatalie]

I was able to compile the shader successfully with the options given:

test.hlsl:

Texture2D video_image : register(t0, space0);
SamplerState video_sampler : register(s0, space0);

float4 main(float2 input_uv : TEXCOORD0) : SV_TARGET
{
    float2 uv = input_uv;
    float4 colourr = video_image.Sample(video_sampler, uv);
    return colourr;
}

$ dxc -Zi -E main -Od -T ps_6_3 -nologo -spirv -fspv-target-env=vulkan1.2 test.hlsl
; SPIR-V
; Version: 1.5
; Generator: Google spiregg; 0
; Bound: 38
; Schema: 0
               OpCapability Shader
               OpMemoryModel Logical GLSL450
               OpEntryPoint Fragment %main "main" %in_var_TEXCOORD0 %out_var_SV_TARGET %video_image %video_sampler
               OpExecutionMode %main OriginUpperLeft
          %6 = OpString "test.hlsl"
               OpSource HLSL 630 %6 "Texture2D video_image : register(t0, space0);
SamplerState video_sampler : register(s0, space0);

float4 main(float2 input_uv : TEXCOORD0) : SV_TARGET
{
    float2 uv = input_uv;
    float4 colourr = video_image.Sample(video_sampler, uv);
    return colourr;
}"
               OpName %type_2d_image "type.2d.image"
               OpName %video_image "video_image"
               OpName %type_sampler "type.sampler"
               OpName %video_sampler "video_sampler"
               OpName %in_var_TEXCOORD0 "in.var.TEXCOORD0"
               OpName %out_var_SV_TARGET "out.var.SV_TARGET"
               OpName %main "main"
               OpName %param_var_input_uv "param.var.input_uv"
               OpName %src_main "src.main"
               OpName %input_uv "input_uv"
               OpName %bb_entry "bb.entry"
               OpName %uv "uv"
               OpName %colourr "colourr"
               OpName %type_sampled_image "type.sampled.image"
               OpModuleProcessed "dxc-commit-hash: 20fefeb1"
               OpModuleProcessed "dxc-cl-option: test.hlsl -E main -T ps_6_3 -Zi -Od -spirv -fspv-target-env=vulkan1.2 -Qembed_debug"
               OpDecorate %in_var_TEXCOORD0 Location 0
               OpDecorate %out_var_SV_TARGET Location 0
               OpDecorate %video_image DescriptorSet 0
               OpDecorate %video_image Binding 0
               OpDecorate %video_sampler DescriptorSet 0
               OpDecorate %video_sampler Binding 0
      %float = OpTypeFloat 32
%type_2d_image = OpTypeImage %float 2D 2 0 0 1 Unknown
%_ptr_UniformConstant_type_2d_image = OpTypePointer UniformConstant %type_2d_image
%type_sampler = OpTypeSampler
%_ptr_UniformConstant_type_sampler = OpTypePointer UniformConstant %type_sampler
    %v2float = OpTypeVector %float 2
%_ptr_Input_v2float = OpTypePointer Input %v2float
    %v4float = OpTypeVector %float 4
%_ptr_Output_v4float = OpTypePointer Output %v4float
       %void = OpTypeVoid
         %17 = OpTypeFunction %void
%_ptr_Function_v2float = OpTypePointer Function %v2float
         %24 = OpTypeFunction %v4float %_ptr_Function_v2float
%_ptr_Function_v4float = OpTypePointer Function %v4float
%type_sampled_image = OpTypeSampledImage %type_2d_image
%video_image = OpVariable %_ptr_UniformConstant_type_2d_image UniformConstant
%video_sampler = OpVariable %_ptr_UniformConstant_type_sampler UniformConstant
%in_var_TEXCOORD0 = OpVariable %_ptr_Input_v2float Input
%out_var_SV_TARGET = OpVariable %_ptr_Output_v4float Output
       %main = OpFunction %void None %17
         %18 = OpLabel
%param_var_input_uv = OpVariable %_ptr_Function_v2float Function
         %21 = OpLoad %v2float %in_var_TEXCOORD0
               OpStore %param_var_input_uv %21
         %22 = OpFunctionCall %v4float %src_main %param_var_input_uv
               OpStore %out_var_SV_TARGET %22
               OpLine %6 9 1
               OpReturn
               OpFunctionEnd
               OpLine %6 4 1
   %src_main = OpFunction %v4float None %24
               OpLine %6 4 20
   %input_uv = OpFunctionParameter %_ptr_Function_v2float
               OpNoLine
   %bb_entry = OpLabel
         %uv = OpVariable %_ptr_Function_v2float Function
    %colourr = OpVariable %_ptr_Function_v4float Function
               OpLine %6 6 17
         %30 = OpLoad %v2float %input_uv
               OpLine %6 6 12
               OpStore %uv %30
               OpLine %6 7 22
         %31 = OpLoad %type_2d_image %video_image
               OpLine %6 7 41
         %32 = OpLoad %type_sampler %video_sampler
               OpLine %6 7 56
         %33 = OpLoad %v2float %uv
               OpLine %6 7 22
         %35 = OpSampledImage %type_sampled_image %31 %32
         %36 = OpImageSampleImplicitLod %v4float %35 %33 None
               OpLine %6 7 12
               OpStore %colourr %36
               OpLine %6 8 12
         %37 = OpLoad %v4float %colourr
               OpLine %6 8 5
               OpReturnValue %37
               OpFunctionEnd

Since it sounds like compilation succeeds but there's unexpected results at runtime, we're unfortunately not able to process your issue with the information provided. If you provide the known-good SPIR-V compiled from GLSL and can identify something in the diff that you expect is causing the issue, then we could investigate further. In general, if the SPIR-V produced is legal and valid, we don't have the ability to debug application errors without a repro case and an expected result. (Closing this for now but happy to reopen if more information is provided.)

[BattleAxeVR]

Sure thing, I'll re-test using the latest DXC and glslLangvalidator and provide the compiled output of each one to compare.

[BattleAxeVR]

I just re-tested today, latest DXC, Vulkan video, everything. Same issue. I'm basically stuck using GLSL / glslLangvalidator to decompress / copy my video textures before I can sample them.

https://github.com/nvpro-samples/vk_video_samples/issues/60

I'm not getting colours from DXC from an immutable texture sampler over YUV planar textures, it's red (1 channel) only. Clearly it's not able to convert YUV 420 -> RGB automatically, like it's supposed to.

[sudonatalie]

We don't build and run full graphics stack applications as part of diagnosing compiler issues, so we can't debug an issue like this on our end with the information provided. We can't even say for sure that this is a compiler bug, since the artifacts you're seeing could be caused by a driver defect or a bug somewhere else in the application.

Can you provide a minimal sample of HLSL that reproduces the issue when compiled with DXC, and a minimal sample of SPIR-V that works as expected? If we can compare SPIR-V that's working as expected against what DXC is producing we might be able to identify a specific compiler issue.

[BattleAxeVR]

All I need is for the SPIRV bytecode outputted by DXC.exe to give identical values to the SPV from the same GLSL fragment shader.

HLSL:

Texture2D video_texture: register(t0, space0); SamplerState sampler: register(s0, space0);

struct VSOutput { float4 Pos : SV_POSITION; [[vk::location(0)]] float2 UV : TEXCOORD0; };

float4 main(VSOutput input) : SV_TARGET { float4 color = video_texture.Sample(sampler, input.UV); return color; }

GLSL:

version 460

layout(set = 0, binding = 0) uniform sampler2D video_texture;

layout(location = 0) in vec2 outUV; layout(location = 0) out vec4 fragColor;

void main() { vec4 colour = texture(video_texture, outUV); fragColor = colour; }

Are these two shaders, in fact, equivalent?

This is as simple a repro I could possibly come up with. There's nothing complicated about these two shaders. The SPV outputted by both DXC and glslLangValidate should be the same, thus 100% interchangeable if I load either up in binary format, right? If that's not the case, then my apologies.

[BattleAxeVR]

This is the output from DXC (latest release downloaded from here)

dxc -E main -Od -T ps_6_0 -spirv -fspv-target-env=vulkan1.3 video_ps.hlsl

; SPIR-V ; Version: 1.6 ; Generator: Google spiregg; 0 ; Bound: 44 ; Schema: 0 OpCapability Shader OpMemoryModel Logical GLSL450 OpEntryPoint Fragment %main "main" %gl_FragCoord %in_var_TEXCOORD0 %out_var_SV_TARGET %textureColor %samplerColor OpExecutionMode %main OriginUpperLeft OpSource HLSL 600 OpName %type_2d_image "type.2d.image" OpName %textureColor "textureColor" OpName %type_sampler "type.sampler" OpName %samplerColor "samplerColor" OpName %in_var_TEXCOORD0 "in.var.TEXCOORD0" OpName %out_var_SV_TARGET "out.var.SV_TARGET" OpName %main "main" OpName %VSOutput "VSOutput" OpMemberName %VSOutput 0 "Pos" OpMemberName %VSOutput 1 "UV" OpName %param_var_input "param.var.input" OpName %src_main "src.main" OpName %input "input" OpName %bb_entry "bb.entry" OpName %color "color" OpName %type_sampled_image "type.sampled.image" OpDecorate %gl_FragCoord BuiltIn FragCoord OpDecorate %in_var_TEXCOORD0 Location 0 OpDecorate %out_var_SV_TARGET Location 0 OpDecorate %textureColor DescriptorSet 0 OpDecorate %textureColor Binding 0 OpDecorate %samplerColor DescriptorSet 0 OpDecorate %samplerColor Binding 0 %int = OpTypeInt 32 1 %int_1 = OpConstant %int 1 %float = OpTypeFloat 32 %type_2d_image = OpTypeImage %float 2D 2 0 0 1 Unknown %_ptr_UniformConstant_type_2d_image = OpTypePointer UniformConstant %type_2d_image %type_sampler = OpTypeSampler %_ptr_UniformConstant_type_sampler = OpTypePointer UniformConstant %type_sampler %v4float = OpTypeVector %float 4 %_ptr_Input_v4float = OpTypePointer Input %v4float %v2float = OpTypeVector %float 2 %_ptr_Input_v2float = OpTypePointer Input %v2float %_ptr_Output_v4float = OpTypePointer Output %v4float %void = OpTypeVoid %20 = OpTypeFunction %void %VSOutput = OpTypeStruct %v4float %v2float %_ptr_Function_VSOutput = OpTypePointer Function %VSOutput %30 = OpTypeFunction %v4float %_ptr_Function_VSOutput %_ptr_Function_v4float = OpTypePointer Function %v4float %_ptr_Function_v2float = OpTypePointer Function %v2float %type_sampled_image = OpTypeSampledImage %type_2d_image %textureColor = OpVariable %_ptr_UniformConstant_type_2d_image UniformConstant %samplerColor = OpVariable %_ptr_UniformConstant_type_sampler UniformConstant %gl_FragCoord = OpVariable %_ptr_Input_v4float Input %in_var_TEXCOORD0 = OpVariable %_ptr_Input_v2float Input %out_var_SV_TARGET = OpVariable %_ptr_Output_v4float Output %main = OpFunction %void None %20 %21 = OpLabel %param_var_input = OpVariable %_ptr_Function_VSOutput Function %25 = OpLoad %v4float %gl_FragCoord %26 = OpLoad %v2float %in_var_TEXCOORD0 %27 = OpCompositeConstruct %VSOutput %25 %26 OpStore %param_var_input %27 %28 = OpFunctionCall %v4float %src_main %param_var_input OpStore %out_var_SV_TARGET %28 OpReturn OpFunctionEnd %src_main = OpFunction %v4float None %30 %input = OpFunctionParameter %_ptr_Function_VSOutput %bb_entry = OpLabel %color = OpVariable %_ptr_Function_v4float Function %35 = OpLoad %type_2d_image %textureColor %36 = OpLoad %type_sampler %samplerColor %38 = OpAccessChain %_ptr_Function_v2float %input %int_1 %39 = OpLoad %v2float %38 %41 = OpSampledImage %type_sampled_image %35 %36 %42 = OpImageSampleImplicitLod %v4float %41 %39 None OpStore %color %42 %43 = OpLoad %v4float %color OpReturnValue %43 OpFunctionEnd

This is the output from glslLangValidator on the above GLSL fragment shader:

glslangValidator.exe video.frag --target-env vulkan1.3 -H

video.frag // Module Version 10600 // Generated by (magic number): 8000b // Id's are bound by 23

                          Capability Shader
           1:             ExtInstImport  "GLSL.std.450"
                          MemoryModel Logical GLSL450
                          EntryPoint Fragment 4  "main" 13 17 21
                          ExecutionMode 4 OriginUpperLeft
                          Source GLSL 460
                          Name 4  "main"
                          Name 9  "colour"
                          Name 13  "video_texture"
                          Name 17  "outUV"
                          Name 21  "fragColor"
                          Decorate 13(video_texture) DescriptorSet 0
                          Decorate 13(video_texture) Binding 0
                          Decorate 17(outUV) Location 0
                          Decorate 21(fragColor) Location 0
           2:             TypeVoid
           3:             TypeFunction 2
           6:             TypeFloat 32
           7:             TypeVector 6(float) 4
           8:             TypePointer Function 7(fvec4)
          10:             TypeImage 6(float) 2D sampled format:Unknown
          11:             TypeSampledImage 10
          12:             TypePointer UniformConstant 11

13(video_texture): 12(ptr) Variable UniformConstant 15: TypeVector 6(float) 2 16: TypePointer Input 15(fvec2) 17(outUV): 16(ptr) Variable Input 20: TypePointer Output 7(fvec4) 21(fragColor): 20(ptr) Variable Output 4(main): 2 Function None 3 5: Label 9(colour): 8(ptr) Variable Function 14: 11 Load 13(video_texture) 18: 15(fvec2) Load 17(outUV) 19: 7(fvec4) ImageSampleImplicitLod 14 18 Store 9(colour) 19 22: 7(fvec4) Load 9(colour) Store 21(fragColor) 22 Return FunctionEnd

As you can see, the outputs are completely different, and when saved to SPV files, the HLSL -> DXC is 2k on disc, while the GLSL is 1k. There is just no way these are even remotely similar, and I'm at a loss to understand what's happening and why. Let alone how to fix it so they give the same results in terms of sampling a texture from an immutable sampler

[sudonatalie]

I'm not familiar with the SPIR-V format pasted above, but I ran your GLSL shader through glslang in Shader Playground (link) to get:

; SPIR-V
; Version: 1.6
; Generator: Khronos Glslang Reference Front End; 11
; Bound: 23
; Schema: 0
               OpCapability Shader
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical GLSL450
               OpEntryPoint Fragment %main "main" %video_texture %outUV %fragColor
               OpExecutionMode %main OriginUpperLeft
               OpSource GLSL 460
               OpName %main "main"
               OpName %colour "colour"
               OpName %video_texture "video_texture"
               OpName %outUV "outUV"
               OpName %fragColor "fragColor"
               OpDecorate %video_texture DescriptorSet 0
               OpDecorate %video_texture Binding 0
               OpDecorate %outUV Location 0
               OpDecorate %fragColor Location 0
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
      %float = OpTypeFloat 32
    %v4float = OpTypeVector %float 4
%_ptr_Function_v4float = OpTypePointer Function %v4float
         %10 = OpTypeImage %float 2D 0 0 0 1 Unknown
         %11 = OpTypeSampledImage %10
%_ptr_UniformConstant_11 = OpTypePointer UniformConstant %11
%video_texture = OpVariable %_ptr_UniformConstant_11 UniformConstant
    %v2float = OpTypeVector %float 2
%_ptr_Input_v2float = OpTypePointer Input %v2float
      %outUV = OpVariable %_ptr_Input_v2float Input
%_ptr_Output_v4float = OpTypePointer Output %v4float
  %fragColor = OpVariable %_ptr_Output_v4float Output
       %main = OpFunction %void None %3
          %5 = OpLabel
     %colour = OpVariable %_ptr_Function_v4float Function
         %14 = OpLoad %11 %video_texture
         %18 = OpLoad %v2float %outUV
         %19 = OpImageSampleImplicitLod %v4float %14 %18
               OpStore %colour %19
         %22 = OpLoad %v4float %colour
               OpStore %fragColor %22
               OpReturn
               OpFunctionEnd

Compared to the DXC output, there are small but notable differences in the image type's depth attribute and how explicitly the sampler types are defined. I suspect this is due to differences in the default behaviors of image sampling between HLSL and GLSL. That doesn't necessarily mean either is incorrect though. HLSL and GLSL are different high level languages with different semantics. The goal of the SPIR-V backend in DXC is to accurately represent the meaning of the HLSL in SPIR-V. It's outside of our scope to help with translation of GLSL to HLSL. That being said, if you want my guess:

Clearly it's not able to convert YUV 420 -> RGB automatically, like it's supposed to.

Do you have any reference to an HLSL or D3D spec that says this is supposed to happen automatically? I think this might be an incorrect assumption. A quick search reveals a number of sample HLSL shaders where that colorspace conversion is done explicitly in the shader, so you might want to start there.

[BattleAxeVR]

https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_sampler_ycbcr_conversion.html

It's part of the official spec and works perfectly in my GLSL shader directly sampling the texture through the immutable sampler (the C/C++ side code hasn't changed in my example, and nor should it. If I just change the fragment / pixel shader it should just work, assuming the bindings are correctly set, which is what I'm trying to figure out).

It's not just DXC.exe manifesting this issue with my utterly basic / trivial HLSL pixel shader here, slang-d with direct spirv output gives me the same result. So I suspect it is something wrong with my shader but I've migrated dozens of shaders from GLSL to HLSL and this is literally the simplest one that works perfectly for all other use cases except for ycbcr video textures.

[BattleAxeVR]

I will attempt to modify ffplay from the latest ffmpeg build that uses vulkan video, to consume an HLSL -> DXC or HLSL -> glslLangvalidator compiled example. I'm somewhat convinced now that the issue is in the HLSL semantics or syntax or shader bindings somewhere, but I can't figure it out yet. It may not be a bug in DXC at all, but HLSL may not even be capable of doing this whatsoever. For curiosity's sake, I also ran my original GLSL fragment shader (that works fine), through a GLSL -> HLSL source converter, then compiled that using DXC, and still the same thing. Now, there may be a bug in the converter program, but I kind of doubt it wouldn't work properly for this extremely simple pixel shader. All it does is sample the texture and output the value, that's it. Nothing simpler than that except outputting a solid colour, really.

[sudonatalie]

VK_KHR_sampler_ycbcr_conversion is a Vulkan extension, not part of the core spec, and it is not currently implemented by DXC. You may submit a feature request for support for this extension to be added to DXC, but I will caveat that it is not likely to be a high-priority enhancement right now.

I'm going to close this issue because translating high level language shaders is outside of this repository's scope and there isn't any indication of a bug in DXC here. If I were you though, I'd try out some of the examples available online when you search for HLSL shaders with explicit YUV to RGB colorspace conversion.

[BattleAxeVR]

slang has a workaround, Sampler2D, that solved it.

https://github.com/shader-slang/slang/issues/3443

I would definitely like to start the process to request this feature in DXC so I can use official HLSL code.

@sudonatalie, can you tell me how one might go about that?

It's totally fine if it takes a while, but I'd like to get the ball rolling.

[sudonatalie]

To clarify, when I've suggested above that you make the conversion explicit in the shader, I'm saying that I believe the functionality you're looking for can already be done by DXC using standard HLSL -> SPIR-V if you add code to your shader that expressly does the conversion from the YUV colorspace to RGB because it doesn't happen automatically. As far as I know, this is how colorspace conversion is done for DirectX, and there are many examples online of HLSL shaders with YUV to RGB conversion.

If you want to propose a new language feature to support automatic colorspace conversion, or you have questions about the HLSL spec's intended behavior in this case, you can file an issue at: https://github.com/microsoft/hlsl-specs/issues/new/choose. This is a non-trivial process and requires a strong justification for acceptance.

But again, from everything you've said here, it doesn't sound like you need a compiler or spec change. You may just need to rewrite your GLSL shader (with some implicit colorspace conversion) in HLSL (which I think requires explicit colorspace conversion in this case).

[BattleAxeVR]

In Vulkan Video, as far as I know, we do not have access to the Y and UV planes individually. @zlatinski, do you know the answer to this question?

Explicit Y and UV plane access is something I would like to have anyway for some situations (like single-channel videos containing ex depth or alpha, where I'd rather no conversion or manipulation done whatsoever anyway).

Back in 2016 I did exactly what you propose, manually convert the separate Y and UV planes back to RGB in the shader, and it was messy. We had colourspace conversion issues requiring us to pass in constants to the shader to match the metadata in the video file, which immutable sampler approach in Vulkan Video handles for us, inside the display driver itself.

As far as I understand it, all DXC needs to provide here for this to work is a Sampler2D equivalent from slang, similar the old school DX9 sampler2d / tex2d texture sampling code.

From what I understand from the slang link above, I believe recent / modern HLSL is simply unable to use immutable samplers due to separation of Texture2D and SamplerState, instead of a single Sampler2D which uses a combined sampler. I do not wish to do manual colourspace conversions when the display driver supports this already. I don't believe DXC would need to implement anything there either (I could be totally wrong), just make Sampler2D work again. I say "just" but this may require some work, but I'd like to request that. Manual colourspace conversion isn't something I intend to do. I am already using an ifdef now to switch between compiling my valid HLSL code (which compiles with DXC) and the slang-specific Sampler2D, but it's awkward and I worry there may be issues with slang I did not anticipate.

To be clear, I'm using these video textures to apply direct and indirect lighting in my path tracing game and I don't want to burden my raygen shader with additional shader instructions for conversion, even if I did have access to those individual planes directly, nor do I wish to convert them all to RGB first then sample those, which doubles their memory footprint. (420 -> RGB). And some of these video textures are very large and high framerate (120 FPS), ex environment map / latlong animated sky is 6K or 8K 120 FPS. It's a lot of extra VRAM bandwidth wasted to convert all this in an extra GPU copy. The whole point of GPU decoding is zero copy and direct application in a shader, for simplicity, memory and performance.

@csyonghe If you don't mind, can you confirm whether it's true that the main issue here is simply HLSL doesn't support combined texture samplers? This would be an API change, right? I understand API changes in HLSL is a big ask, but if there's another way to do this, I'm all ears.

I would be very, very happy to be wrong, but I'd rather not have no choice but to use slang. I like having options in case there are unforeseen bugs. If I have to switch my entire pipeline to use slang-specific HLSL code, I will, but it's a bit risky for me. What if I run into other bugs in the future that only exist in slangc but not DXC?

Also, as a practical consideration, DXC has first-class integration inside Visual Studio, which I absolutely must have to remain productive while writing shader code, for ex: when there's a compile error, I click on the offending line in the output window and it takes me straight to it. It's one thing using using slang for its other benefits, but quite another to lose productivity by having to compile my shaders offline, find the line, etc. Or switch to VS Code after I have 25 years of VS experience and muscle memory.

Is there no way to convert YUV planar video to RGB in DirectX / HLSL? There must be by now. I doubt anyone wants to pollute their nice, simple shader code with highly domain-specific code that requires significant time investment and overhead to understand all the ins and outs of colourspace conversion. I worked with many colorists / pros in this area at Felix and Paul Studios (VR video pioneers) and even then, it took significant combined group effort to debug back in 2016.

[sudonatalie]

I'm not sure if it would fix this issue, but the SPIR-V backend in DXC does support combined samplers via a Vulkan-specific annotation in HLSL. See: https://github.com/microsoft/DirectXShaderCompiler/wiki/Vulkan-combined-image-sampler-type

[BattleAxeVR]

Wow, that looks perfect! I will try that immediately. Thanks so much Natalie. It looks like the actual solution to this problem.

I was also going to look into the feasibility of whether it's possible to use explicitly bound samplers instead of combined or at least immutable ones, that have the ycbcr conversion stuff added to the pnext structs in Vulkan, but this looks like a potential real fix. I do not want to lose the ability to compile with DXC, even if slang does provide some interesting advantages especially for path tracing + machine learning.

[csyonghe]

@BattleAxeVR glad to see that your issue can also be addressed without moving away from dxc. Just in case you decided to use Slang anytime in the future, we are committed to support your use cases and address any issues you run into.

[BattleAxeVR]

Thanks everyone, I now have a working fix for both DXC and slang pipelines, giving me the flexibility I need. You're the best! Have a great weekend, everyone.

I definitely plan on using Slang for path tracing eventually, starting with Nate's workaround for the other DXC issue we had in anyhit shaders where it fails to copy out the ray payload if the ray is terminated (to allow me to manually do noise-free alpha compositing across multiple bounces w/o introducing noise that current denoisers like NRD do not handle properly)

This is the command line for DXC I used, in case it helps anyone else "out there":

dxc -Zi -E main -Od -T ps_6_6 -nologo -spirv -fspv-target-env=vulkan1.3 video_pixel_shader.frag.hlsl /E"main" /Fo"video_pixel_shader.spv" /nologo

[[vk::combinedImageSampler]] Texture2D textureColor : register(t0);

[[vk::combinedImageSampler]] SamplerState samplerColor : register(t0);

struct VSOutput { [[vk::location(0)]] float2 UV : TEXCOORD0; };

float4 main(VSOutput input) : SV_TARGET { float4 colour = textureColor.Sample(samplerColor, input.UV); return colour; }

As Natalie's example shows, all that's needed for DXC to handle this case is the new [[vk::combinedImageSampler]] decorator in front of both the Texture2D and SamplerState, the rest of the HLSL code is exactly the same so I don't need to wrap sampling calls in a function using an ifdef for slang vs dxc compilation.

[sudonatalie]

🔥