platform differences in rendering openXRCompositionLayers with Hole Punch feature

[surreal6]

Tested versions

• reproductible in 4.3.beta2
• godot openXR vendors 3.0.0

System information

Godot v4.3.beta2 - Windows 10.0.19045 - GLES3 (Compatibility) - NVIDIA GeForce RTX 2080 SUPER (NVIDIA; 31.0.15.3640) - AMD Ryzen 9 3900X 12-Core Processor (24 Threads)

Issue description

Testing OpenXRCompositionLayers I found some differences in rendering on diferent platforms for CompositionLayerEquirectangular and CompositionLayerQuad when using the Hole Puch feature.

SteamVR is not able to render the Hole Punch but Quest2 is able to do it.

There is a demo project where we discuss the bug. As @BastiaanOlij pointed in the demo project comment, seems to be a problem on the XR runtime.

https://github.com/godotengine/godot-demo-projects/pull/1080#issuecomment-2198520700

Steps to reproduce

I Attach a modified version of Bastiaan demo. Add android vendors and compare SteamVR and Quest2 export results.

Please see this comment where I explain how to reproduce it and see some images

https://github.com/godotengine/godot-demo-projects/pull/1080#issuecomment-2198520700

Minimal reproduction project (MRP)

openxr-composition-layers.zip

[dsnopek]

Sorry for the delay in getting back to this!

I tested your MRP on Windows with the Oculus runtime, and hole punching worked: the meshes attached to my hands were rendered on top of the quad, as expected.

However, when I tried with the SteamVR runtime, they were rendered behind the quad.

(FYI, the reason the equirect doesn't have these problems is because equirect composition layers aren't supported by OpenXR runtime, so we're emulating it in Godot.)

I don't yet have any good theories as to what could be going wrong when using SteamVR.

[dsnopek]

After a little more experimentation, it seems like SteamVR's compositor is always drawing the projection composition layer first, and the quad composition layer second, regardless of the order we provide to OpenXR's xrEndFrame().

I'll add some debug print statements when I have a chance (I've been testing with Godot 4.3-beta3) in order to verify that we're providing the layers in the correct order, however, I can't imagine we'd sort them any differently depending on which OpenXR runtime is being used.

If SteamVR is disregarding the order we're providing the composition layers, then it means it's not conforming to the OpenXR spec:

See https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#compositing

[dsnopek]

I tried adding some debug code in OpenXRAPI::end_frame() that loops over the array of composition layers we're sending to xrEndFrame() and prints out the type on the struct.

Debug diff

``` diff --git a/modules/openxr/openxr_api.cpp b/modules/openxr/openxr_api.cpp index 541e369925..895283f71c 100644 --- a/modules/openxr/openxr_api.cpp +++ b/modules/openxr/openxr_api.cpp @@ -2367,8 +2367,11 @@ void OpenXRAPI::end_frame() { // Now make a list we can pass on to OpenXR. Vector layers_list; + int iii = 0; for (OrderedCompositionLayer &ordered_layer : ordered_layers_list) { layers_list.push_back(ordered_layer.composition_layer); + print_line("Composition layer ", itos(iii), " is ", itos(ordered_layer.composition_layer->type)); + iii++; } XrFrameEndInfo frame_end_info = { ```

And both when running with the Oculus and SteamVR runtime, I get:

Composition layer  0  is  36
Composition layer  1  is  35

This confirms that it's passing the quad first and then the projection layer, which means it should draw the quad under the projection layer, but it's not.

So, I guess this is a bug in SteamVR! I'll see if I can find a place to report it to Valve.

UPDATE: I've posted a note about the bug in the "Bug Reports" discussion section for SteamVR: https://steamcommunity.com/app/250820/discussions/3/4520009262276938902/

[dsnopek]

In searching through SteamVR bug reports, I spotted this post from about a year ago:

See https://steamcommunity.com/app/250820/discussions/3/3843305689135170424/#c3820788154612824924

They say, "We do treat quad layers as an OpenVR VROverlay."

Perhaps they are mapping OpenXR quad layers to this other concept which isn't capable of being drawn underneath the projection layer? I'm not personally familiar with OpenVR at all.