Behavior is not defined when VkColorSpaceKHR conflicts with vkSetHdrMetadataEXT

[zehui-l]

When choosing a color space enabled by VK_EXT_swapchain_colorspace, the VkColorSpaceKHR defines the values of red/green/blue primary and white-point.

In another extension vkSetHdrMetadataEXT, I can specify the VkHdrMetadataEXT which also includes the red/green/blue primary and white-point.

The behavior is not defined when I set custom values in VkHdrMetadataEXT that are different from those described in VkColorSpaceKHR. Which extension has higher priority?

[linyaa-kiwi]

I inspected the Vulkan spec, but failed to find where it defined the interaction between VkColorSpaceKHR and VkHdrMetadataEXT. I discovered a related ambiguity: the spec does not clearly define VK_COLOR_SPACE_PASS_THROUGH_EXT.

I will raise this question for discussion inside the weekly Khronos System Integration meeting.

For reference, can you share which drivers and operating systems on which you see VK_EXT_hdr_metadata supported? It is not supported in the drivers that I work with (Mesa drivers on Linux).

[zehui-l]

For reference, can you share which drivers and operating systems on which you see VK_EXT_hdr_metadata supported? It is not supported in the drivers that I work with (Mesa drivers on Linux).

I tested NVIDIA Quadro RTX 6000 (driver 537.58) on Windows 10 and Qualcomm Adreno 730 on Android. I check gpuinfo.org and it looks like most new NVIDIA/AMD/Intel/Qualcomm devices support the VK_EXT_hdr_metadata extension. I also notice that the Unity URP HDR pipeline calls these APIs.

[ppaalanen]

VkColorSpaceKHR primaries define the color encoding, that is, it defines what colorimetry the RGB values correspond to.

HDR static metadata primaries define the target color gamut, that is, the gamut that is expected/intended to contain the meaningful content. Actually they are defined in SMPTE ST 2086:2018 as the mastering display color volume (MDCV), which tells the color gamut and luminance range that the colorist was able to see on their display while color grading the content. Anything outside of what they were able to see is presumed to not exist in the content or at least not have an intended reproduction.

These two concepts are completely orthogonal, and there is no "priority" between them. They are just different things.

If I were to draw an analogy, let's think of real numbers. We can use IEEE 754 to encode real numbers as digital representations. This is the encoding. But if I am not expecting all encodable real numbers, I might say that I use only the numbers between 5.5 and 11.0. That is the target range. Next, if I want to normalize real numbers into integers [0, 1023], how would I do that? If I take the whole encodable range, I'm lucky to get even one integer assigned for my range of interest. It basically destroys the data. But, if I take my range of interest, then all the integers [0, 1023] will be meaningful to me, but I also lose the ability to represent anything outside. Then again, I do not care of anything outside.

See:

• container color volume
• target color volume
• primary color volume

The problem with HDR static metadata primaries is that they were originally intended to be sent to the sink (display) as metadata, and then the sink was supposed to do something smart with it when adapting the imagery to its colorimetric capabilities. Unfortunately, ignoring that metadata completely is a valid implementation, so lots of displays do just that.

This means that if you play with the HDR static metadata values, you might not see any difference in practice on the OS and display you use.

I should add that some Wayland compositors intend to make good use of that metadata, so think carefully what you put in it. It is better to not put anything rather than put something that is wrong.

[linyaa-kiwi]

From today's Khronos System Integration meeting:

• We should add clarifying text to the Vulkan spec to remove confusion about the complementary purposes of VkColorSpaceKHR and VkHdrMetadataEXT.

• @swick has more clarification about colorspaces to share from the SMPTE.

[swick]

This is something that was on my TODO for a long time now. Created an MR at the color-and-hdr repo (https://gitlab.freedesktop.org/pq/color-and-hdr/-/merge_requests/41) which tries to explain what the metadata is for and how it's supposed to be used. It also includes the document we received from SMPTE.

[zehui-l]

I think it would be good to add some recommendations for game developers. If I understant it correctly, the static HDR metadata should only be modified by the colorists in game editors. And when the games are released and distributed, the static HDR metadata should always remain as it is in the editor.

[ppaalanen]

I think it would be good to add some recommendations for game developers. If I understant it correctly, the static HDR metadata should only be modified by the colorists in game editors. And when the games are released and distributed, the static HDR metadata should always remain as it is in the editor.

Yes, and even further: the HDR static metadata (MDCV part) should reflect the display capabilities and setup of the colorist working in a standard-defined viewing environment. HDR static metadata does not communicate the viewing environment, that needs to be communicated somehow else. ITU-R BT.2100 is a possible candidate determining a viewing environment ("Reference viewing environment for critical viewing of HDR programme material"), since it includes PQ and HLG systems.

I see that VkColorSpaceKHR refers to "HDR10". I have not seen anything actually say what the reference viewing environment for HDR10 content is. Hmm, SMPTE ST 2084:2014 has an annex for "Example Reference Viewing Environments (Informative)". I guess everyone just assumes either "dim living room" or "dark movie theater", depending on what their users are likely to be in.

[swick]

Not sure how much information should be added to the specification though. VK_EXT_hdr_metadata already goes into a lot of details.

[zehui-l]

Not sure how much information should be added to the specification though. VK_EXT_hdr_metadata already goes into a lot of details.

One option is to add some introduction in the Khronos Data Format Specification. And the VK_EXT_hdr_metadata refers to it, like the VkColorSpaceKHR does.

[ppaalanen]

VK_EXT_hdr_metadata does seem to confuse reference monitor and mastering display, though, at least it's consistent.

[linyaa-kiwi]

Update. Khronos has a few internal MRs under review to resolve this.

[oddhack]

This should be fixed in the 1.3.290 spec update. If you do not agree, please reopen with additional context.