SteamVR applies unnecessary linear-to-sRGB conversion to all swapchain images supplied via OpenXR

[amini-allight]

This bug report is for SteamVR's OpenXR runtime functionality rather than OpenVR, but I have been instructed to post such bug reports here by a Valve employee in the past.

I discovered while working on my project that known colors passed to SteamVR via OpenXR don't look correct (they are much brighter) when viewed either through the physical headset or the "Display VR View" option within SteamVR. This can be fixed by applying an sRGB-to-linear conversion function to the (already linear) color values prior to submitting them to SteamVR. The fact that this fix works suggests the issue is SteamVR doing an unnecessary linear-to-sRGB conversion on supplied images, which the fix counteracts. This issue occurs regardless of whether the swapchain images are configured to use an sRGB format like VK_FORMAT_R8G8B8A8_SRGB or a linear format like VK_FORMAT_R16G16B16A16_SFLOAT, suggesting it is an issue with general OpenXR swapchain image handling rather than an issue specifically with treating sRGB images as linear ones.

I have uploaded a program that demonstrates this bug in a repository on my GitLab and more information can be found in the readme over there.

I also found this bug report from last year which seems to be describing the same issue.

Platform info:

• Valve Index headset
• AMD RX 6900 XT
• Linux 6.5.2
• Mesa 23.1.7
• SteamVR 1.27.5

[amini-allight]

Tested this on the SteamVR 2.0.1 beta today, bug is present in that version too.

[shinyquagsire23]

Been working on HDR support for ALVR and can confirm this is still the case for non-SRGB formats (RGBA float16) on SteamVR 2.4.3 on my Index. My gold standard target for HDR tests on ALVR has been a combination of OpenBrush and SpecialK, which can force the framebuffers to float16 instead of RGBA8_SRGB (normally used for flat HDR displays for people who want less banding). The skybox is a very dark gray when shown correctly (and has noticeable banding on 8-bit displays). On my Index, it displays as a very washed out light gray.

Admittedly I only tested this because I was curious if I should or should not be doing sRGB corrections on linear RGB layers received by the ALVR driver since it's a bit ambiguous (storage vs shader input vs shader output colors get extra messy with encoders involved).

Actually, if it's happening regardless of the input format, that would also explain why two titles had really odd behavior when I was testing with SpecialK: Bonelab and Beat Saber both are OpenXR apps, both output RGBA8_SRGB by default, and both required a very strange extra linear-to-sRGB pass in ALVR iff their swapchains are forced to RGBA float16. Which would imply that Unity might have a SteamVR-specific OpenXR hackfix for this now, yikes.

[shinyquagsire23]

Honestly the more I look at this, the more confused I get. On an Index, I get the following results:

• hellovr_dx12, as compiled from this repo as of today, renders washed out colors in the headset, and correctly in the window (especially noticeable on the controller models)
• hellovr_dx12, with this diff to use float16 framebuffers, renders colors correctly
• Beat Saber and Bonelab, with framebuffers forced to DXGI_FORMAT_R16G16B16A16_FLOAT using SpecialK, renders correctly on Index (in reality, the linear colors are too dark)
• OpenBrush, with framebuffers forced to DXGI_FORMAT_R16G16B16A16_FLOAT using SpecialK, renders washed out colors.
• OpenBrush, running normally, renders correctly despite being an OpenXR app.
• hello_xr from this repo in GL_RGBA16F swapchain format (the default for the sample) renders too bright.
• hello_xr from this repo in GL_SRGB8_ALPHA8 swapchain format renders correctly.

Edit: Gave it a think. I think that

• hellovr_dx12 is just borked and is rendering "correctly". So SteamVR layer rendering is fine.
• The OpenXR layer rendering is applying sRGB to every linear texture format (so, an improvement from when this issue was first made), when it shouldn't be. Which means someone at Unity tested OpenXR + float16 framebuffers, saw the sRGB error, and incorrectly added extra processing so that it would look right in SteamVR. But then every game just uses SRGB8 anyway.
• OpenBrush uses some Unity 2021 version which does not have the incorrect extra processing.