YCbCr immutable sampler not working in HLSL via DXC.exe

[BattleAxeVR]

Hi all, I just spent a bunch of time trying to figure out why the colours were wrong in my engine for videos, where they were working before. I fixed all the vulkan validation errors and followed the specs for YCbCr conversion via immutable sampler being set during descriptor set layout / bindings instead of setting it during update, but still no luck.

It turns out, using HLSL and DXC needs some new parameters to make it work, but I can't figure out what it is. I tried a bunch of options in the latest HLSL 2021 build of DXC.exe, and it still gives me the same thing (building to vulkan 1.2 spirv, either to a header or to spv files and loaded from disc).

My old GLSL code using glslLandValidator works though, which is what this sample uses (or rather, builds the shaders live via shaderc).

Is there a way to get the ycbcr sampler to work properly using HLSL -> DXC -> SPIRV workflow?

[zlatinski]

Hi @BattleAxeVR, It looks like this issue has been resolved at the https://github.com/microsoft/DirectXShaderCompiler forum. Can we please close it here, as well? This is not really related to Vulkan Video, in the first place.

[BattleAxeVR]

They closed it without even trying to repro it so I don't consider it solved, but that's ok, I can use GLSL for just this shader as a workaround for now until it does work.

Btw I integrated the latest build and using the latest SDK and it works perfectly with no popup. Thanks for that. There are still a bunch of validation errors in Vulkan but I guess those will disappear when this goes from Beta to mainstream drivers. Have any idea if that's likely to happen this year?

[zlatinski]

This shouldn't have been closed, IMO, there seems to be a fundamental problem between HLSL and Vulkan Video and I think people should be aware of it before they start using VV in their projects.

Hi BattleAxeVR,

One clarification here: Vulkan Video does not require any shaders, at all. What you have experienced here is incompatibility between the Vulkan YCbCr sampler extension and HLSL. It just happens that the Vulkan Video sample app is using the YCbCr sampler for the presentation, but it did not have to. We can still use the fragment shader's path to fetch the individual planes using separate texture views and then do the YCbCr to RGBA conversation within the shader. In NVIDIA's implementation, it would not have made any difference in performance since we are still using the same VV image. All you would need to do is create an image view per plane against the VV-decoded output image, as I have done for the filter I have implemented. I'm currently focusing on the encoder's sample, but I'm planning to upgrade the sample to have the presentation layer fragment shader using just that approach as an alternative.

Best Regards! Tony

[BattleAxeVR]

The issue is solved now, thanks! But seeing a sample with explicit access to the Y and separate UV planes is super useful to me as I have lots of data to pack and can't have any cross-contamination of Y / UV if I want my data to be correct. (I need alpha, depth at full res, but 2D motion vectors and tangent space normals should be fine at half res each, so I could stuff those into UV plane using 420).