Color rendering is device-specific

[hybridherbst]

It seems that currently, there's no consistent way to distinguish which device somone is actually using (independent of the runtime in use). From what I've read there seems to be a consensus that such a behaviour is "by design", and that OpenXR should be beyond the idea of device-specific code paths.

However, I think this fails a reality check, namely, that different devices have vastly different characteristics in terms of colors. Vive, Vive Pro, Quest, Rift S all display very different brights and darks, with nuanced color differences across the supported color space. These brightness and color differences cause issues in business cases, where perceptual color matches are very important for brand identities etc (if a company's midnight blue comes over as black or light blue depending on device, that's an issue for adoption of both XR in general and OpenXR specifically).

This is somewhat related to #54 , but only tangentially; the mobile device space has shown that despite many devices having officially matching color spaces, vendors still bump contrast here and there, change colors, adjust toning, ... in a device-specific way to have the most vibrant colors, the deepest blacks, etc.

While there might be a theoretical solution to this in a color space extensions, for the reasons mentioned above I think it's still very important that there's a clear, correct way to know which device someone is using to correct the experience based on that in case that's needed.

This might be less relevant far in the future when there's thousands of devices and we have the same mess as in the mobile landscape (at which point we'd simply give up and accept defeat), but I'd a) be interested in a solution for today with 5-6 VR devices covering 95% of users and b) how to not get into that future situation of "it's a mess, just accept that colors are wrong".

For reference, when using

• runtime: SteamVR/OpenXR, one cannot differentiate between Quest 1, Quest 2 and Rift (both are "SteamVR/OpenXR : oculus")
• runtime: SteamVR/OpenXR, one cannot differentiate between Vive, Vive Pro, and Vive Index
• runtime: Oculus, one can differentiate between them (Quest 1, Quest 2, Rift)

There's all kinds of hacks and workarounds floating around online, ranging from "differentiate by controller type" to "just check the HIDs on the machine for which device is actually attached" to my personal "favorite", "if it's 80fps it's a Rift, 72fps and aspect of 1.1:1 is a Quest, aspect 1:1 is a Quest".

I hope the above explains why I think better ways and specs for distinguishing between devices are important, especially to properly adjust color perception across business cases and color-relevant areas.

Potential solutions:

• Device ID should be accessible (compare to HID), separate devices shouldn't have the same ID
• Actual strict rules around color spaces for devices to be OpenXR compliant
• give up and just accept that colors don't look the same anywhere

[rpavlik-bot]

An issue (number 1620) has been filed to correspond to this issue in the internal Khronos GitLab (Khronos members only: KHR:openxr/openxr#1620 ), to facilitate working group processes.

This GitHub issue will continue to be the main site of discussion.

[rpavlik]

eek! I would say the underlying issue here is the color rendering: if you're having to detect the device type that you're using, that means we have a gap in the spec.

[rpavlik]

It seems that currently, there's no consistent way to distinguish which device somone is actually using (independent of the runtime in use). From what I've read there seems to be a consensus that such a behaviour is "by design", and that OpenXR should be beyond the idea of device-specific code paths.

However, I think this fails a reality check, namely, that different devices have vastly different characteristics in terms of colors. Vive, Vive Pro, Quest, Rift S all display very different brights and darks, with nuanced color differences across the supported color space. These brightness and color differences cause issues in business cases, where perceptual color matches are very important for brand identities etc (if a company's midnight blue comes over as black or light blue depending on device, that's an issue for adoption of both XR in general and OpenXR specifically).

This is somewhat related to #54 , but only tangentially; the mobile device space has shown that despite many devices having officially matching color spaces, vendors still bump contrast here and there, change colors, adjust toning, ... in a device-specific way to have the most vibrant colors, the deepest blacks, etc.

While there might be a theoretical solution to this in a color space extensions, for the reasons mentioned above I think it's still very important that there's a clear, correct way to know which device someone is using to correct the experience based on that in case that's needed.

This might be less relevant far in the future when there's thousands of devices and we have the same mess as in the mobile landscape (at which point we'd simply give up and accept defeat), but I'd a) be interested in a solution for today with 5-6 VR devices covering 95% of users and b) how to not get into that future situation of "it's a mess, just accept that colors are wrong".

So a similar situation came up here for input as well, and the balance that we struck was the system of interaction profiles and suggested bindings: The app can tell the runtime how they'd like to have things mapped on certain well known controllers (the ones that they test on), and the runtime will only refer to those controllers when talking with the application, but the runtime may internally map to an arbitrary controller as it sees fit, ideally configurable by the user for accessibility and compatibility.

I'm not sure how best to make an analogy here to the color rendering, but I'll bring this up with people who might know.

[hybridherbst]

I'm really not sure if this is "solveable by spec". For the reasons outlined above, unless there's a strict verification step that actually verifies that, by default, without a user changing some setting, a device follows color space spec X, I think there will always be device manufacturers that, on purpose, don't follow the spec, "because user research has shown people like the more vibrant colors" (nearly verbatim from a recent blog post from a headset manufactur on why they decided to not correct a color space mistake they made in the past). In these cases, one needs to be able to at least identify these devices.

I'm totally on the same page in regards to an ideal world, but I don't think there's a plan for extensive spec validation for new devices for OpenXR, or is there?

[rpavlik]

So there is a pretty extensive conformance test suite actually. Not sure how easy it would be to add color reproduction to it, but there are already portions that have to be done by a human tester.

Possibly useful reference I found when trying to find the blog post ;) https://www.mdpi.com/1424-8220/20/19/5658 also https://forums.flightsimulator.com/t/bug-feature-provide-color-space-tone-mapping-cas-shader-strength-and-post-processing-effects-controls-in-vr/345703#color-space-and-display-devices-6

[rpavlik]

Can you reach out to me at openxr-speceditor at khronos org to get a private email chain going? Folks have some questions but we'd rather not mention specific devices, etc. in this issue.