GodotVR / godot_openxr_vendors

Godot 4 wrapper for OpenXR vendors loaders and extensions
MIT License
83 stars 17 forks source link

Add Android Surface swapchain support #111

Open maunvz opened 3 months ago

maunvz commented 3 months ago

This is a draft for adding an AndroidSurfaceLayer object that extends Node3D and does the following:

With this change, a project can use Android Surfaces to display content on OpenXR layers and control them with the Godot scene graph.

TODO: Add an example to the demo project

BastiaanOlij commented 3 months ago

We'll need to see how this overlaps with the logic we're building to core around composition layers. Also seeing these are all KHR extensions it probably belongs in core, not in the vendors plugin.

dsnopek commented 3 months ago

I already have a nearly finished PR for Godot that adds support for composition layers, including XrCompositionLayerQuad. I'll see if I can finish it up quick, or maybe just make a draft PR..

dsnopek commented 3 months ago

Here's my composition layer PR: https://github.com/godotengine/godot/pull/89880

dsnopek commented 3 months ago

I spent some more time looking into this!

If I'm understanding correctly, the XR_KHR_android_surface_swapchain extension allows creating a swap chain where the data drawn onto the swapchain is pulled from an Android Surface. So, if you've got an Android plugin that's drawing to an Android Surface, you could use this extension to display that on an OpenXR composition layer.

In that case, I think it would probably make sense to support this by building on PR https://github.com/godotengine/godot/pull/89880, to allow configuring the composition layer nodes to use an Android Surface rather than a Godot SubViewport as the source for the composition layer.

And, I agree with @BastiaanOlij that this should be done in Godot (rather than 'godot_openxr_vendors') because the core extension is XR_KHR_android_surface_swapchain, which is vendor neutral.

The PR here also uses XR_FB_android_surface_swapchain (a Meta vendor extension), but if we want to keep that bit out of Godot, we'll need to add some hook to Godot that allows GDExtensions to modify the swap chain creation. But that shouldn't be a big change, and we'll likely already need something similar to allow modifying composition layer creation (to support numerous other vendor extensions, like, for example, XR_FB_composition_layer_secure_content).