[TRACKER] OpenXR in Godot 4

BastiaanOlij commented 1 year ago

We've identified several outstanding issues with OpenXR. Just raising this issue to keep track of them.

Core features

[ ] Auto exposure only works in left eye #69000
[x] VRS being made available even when not supported #69091
[ ] Query OpenXR for a density map once supported
[ ] Render order issue #68469 (this is likely caused by the depth buffer issues, see below)
[ ] Only background is being rendered #65864 (this is likely caused by depth buffer issues, see below)
[ ] Action map save issue #68041
[x] Fix various issues around the action map #68528
[x] Fix sky orientation issue #69636
[x] Fix stereo issue in sky #62738

Extensions

[ ] Support implementing OpenXR extensions as GDExtensions #68259
[ ] add support for various composition layers (as extra (2D) Viewports rendering to XRSwapChains)
[ ] add support for Dpad bindings in Action map: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_dpad_binding
[ ] Port performance settings extension from Godot 3 plugin: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_performance_settings
[ ] Implement render models API once core version becomes available
[ ] Implement Metas Scene capture extension: https://github.com/migeran/godot_openxr_meta
[ ] Ensure interfaces avoid multiple inheritance of classes with data members #69694

Depth buffer

OpenXR allows supplying the depth buffer, the XR runtime can use this information to perform more accurate re-projection when framerates are not met. On Quest the depth buffer can be combined with a motion vector buffer to enable application spacewarp, a more advanced version of re-projection. Motion vector support is not yet available on the mobile renderer but solving the issues already found will be paramount to solving this.

[x] Left eye depth buffer is supplied for the right eye #69648
[ ] Mobile renderer is not resolving MSAA depth buffer to final depth buffer
[ ] Clustered renderer fails to resolve MSAA depth buffer to final depth buffer if compute shaders are used, it's not possible to set the VK_IMAGE_USAGE_STORAGE_BIT
[ ] Make depth buffer support optional so it can be turned off when effects are used that break the solution #69654

Vulkan specific

OpenGL specific

[x] Image flip issue when presenting preview image on desktop #69581

DX12 specific

[ ] Implement multiview support
[ ] Add DX12 extension for OpenXR

Misc

[ ] WMR devices over SteamVR are not always working #67398 (possible driver issue with MS)

RadixSeven commented 1 year ago

The two issues linked in the "Depth Buffer" section are marked as merged, and the "Submit Depth Buffer" checkbox exists in 4.0.1.stable. Should the "Depth Buffer" section of this issue be checked off? It looks like we can also check off #68469, #65864, and #62738 under "core features".

BastiaanOlij commented 1 year ago

@RadixSeven there were still a few issues with this. #78550 fixes a major issue, I'm still looking into a few resolve issues when MSAA is being used.

godotengine / godot