Depth Layer Submission

[rbessems]

Currently the depth layer is not being submitted, it is very good practice to submit it. (allowing for better stability and depth based operations)

https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_KHR_composition_layer_depth

[AndrewRichards-Code]

Could this be added to Chapter 5 as a discussion of an extension? Currently, we create depth images for rendering the graphics, but the OpenXR compositor isn't aware of them. In the VR case is fine, but for AR the more data to increase the accuracy of depth compositing the better.

[rbessems]

I'm OK with chapter 5.

[AndrewRichards-Code]

I want to check what guarantees OpenXR makes when calling xrEnumerateSwapchainFormats(). Will it always return a valid depth format for us to use? This post on the Steam Community suggests that there might be no guarantees: https://steamcommunity.com/app/250820/discussions/8/2448217320134593992/

The spec only states that swapchain formats should support R8G8B8A8 and R8G8B8A8 sRGB. https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#swapchain-image-management

The XR_KHR_composition_layer_depth again makes no guarantee that runtime must support depth formats for swapchains, despite requiring the use of a depth swapchain to submit with the XrCompositionLayerProjectionView. https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XrCompositionLayerDepthInfoKHR

For the Meta Quest 2, I was able to get a range of color, hdr color, depth and compressed formats. (I think I copied these all correctly.)

PC
37, 43, 44, 50, 97, 122, 124, 126, 129, 130, 133, 134, 135, 136, 137, 138, 143, 144, 145, 146

Android
29, 43, 9, 16, 37, 10, 17, 24, 38, 15, 70, 77, 84, 91, 71, 78, 85, 92, 74, 81, 88, 95, 75, 82, 89, 96, 76, 83, 90, 97, 100, 2, 8, 147, 149, 151, 148, 150, 152, 153, 155, 154, 156, 157, 159, 161, 163, 165, 167, 169, 171, 173, 175, 177, 179, 181, 183, 158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 124, 125, 126, 129

I am looking to 'backport' the use of depth swapchains to previous chapters to minimize chopping and changing the code. What's the best approach here, and should this issue be raised with the Working Group?

[rvkennedy]

This is likely to affect the schedule given the amount of change vs what we have for Ch 2-4.

[rbessems]

Not sure about guarantees, maybe someone else can talk about that.

Assuming the numbers above are vulkan I spot 126 and 130 which are VK_FORMAT_D32_SFLOAT VK_FORMAT_D32_SFLOAT_S8_UINT respectively.

I handle this the same way I handle environment blend mode. My applications knows how to deal with a preference ordered limited set and picks the first match.

[rbessems]

For OpenGLES I use: GL_DEPTH_COMPONENT32F, GL_DEPTH_COMPONENT24

[rbessems]

Looking at hello_xr the application simply assumes VK_FORMAT_D32_SFLOAT

https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/graphicsplugin_vulkan.cpp#L1008

The application creates the color swapchapchain using OpenXR:

https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/openxr_program.cpp#L657

But the depth chain it creates directly without OpenXR involvement.

https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/graphicsplugin_vulkan.cpp#L929

I've done this part differently as our Runtime returns depth formats, so I also have OpenXR create the depth chain. I'll see if I can get the best practice recommendation from the group.

[rbessems]

The group thinks runtimes should support depth chain as it is truly important for AR use cases and is considering a must for the 1.1 version of the spec. We are also considering updating hello_xr to start submitting the depth chain now unconditionally.

With that in mind, lets create a depth chain using the runtime swapchain creation mechanism and submit it to XR_KHR_composition_layer_depth.

[AndrewRichards-Code]

Use depth swapchain and error out if not supported. Discuss reprojection in text and occlusion is coming - some runtimes do this.

[AndrewRichards-Code]

Commit 1180ff7 added the final text for 5.2 and the code for XrCompositionLayerDepthInfoKHR is now in place with a depth swapchain.