search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.03k stars 21.18k forks source link

GUI does not appear on XRCamera3D #90110

Open ultrasuperpingu opened 7 months ago

ultrasuperpingu commented 7 months ago

Tested versions

Reproductible with 4.3-dev5

System information

Windows 11 - Godot 4.3-dev5 Vulkan (Forward+)

Issue description

GUI (Control) does not appear on the PC screen when using XRCamera3D. I understand why it doesn't appear in the HMD but I see no reason it doesn't show up on the desktop mirror view...

Steps to reproduce

Add a GUI to a XR scene.

Minimal reproduction project (MRP)

Don't think it's necessay to provide a project but I can provide it if needed.

clayjohn commented 7 months ago

CC @BastiaanOlij

I am fairly certain this is intended as the XRCamera3D view is supposed to render as if it is an HMD. The rendering path for XR is different and totally skips over 2D/GUI

ultrasuperpingu commented 7 months ago

CC @BastiaanOlij

I am fairly certain this is intended as the XRCamera3D view is supposed to render as if it is an HMD. The rendering path for XR is different and totally skips over 2D/GUI

That was my theory... But I don't see think it's a good idea to remove the gui on the mirror view. If the engine does not handle this, how can I display a Gui on the mirror view on desktop then (and of course, not in the HMD) ?

But I guess it's more a proposal than a bug if it's by design. Should I have to open a proposal and close this ?

BastiaanOlij commented 7 months ago

Hey @ultrasuperpingu ,

We don't render 2D elements in XR mode because there is no good way to render them in place as there is no depth information and because there is no guarantee headsets have horizontal panels.

On the desktop we just show a copy of the left eye render result.

If you want to do what you want, you'll need a different setup. You can create a subviewport and add your XROrigin3D and XRCamera3D to that, and set that viewport up for XR (so setting use_xr to true and all that). This will handle output to the HMD.

You can now use a TextureRect with the texture set to a ViewportTexture pointing to the subviewport. This will now show the left eye. You do need to play around with sizing and positioning.

You can now also add your GUI elements as per normal.

ultrasuperpingu commented 7 months ago

@BastiaanOlij As I understand your advice correctly, the GUI will appear in the HMD no? I'm looking to display a gui on the desktop mirror but not in the HMD. Is it possible ?

BastiaanOlij commented 7 months ago

@BastiaanOlij As I understand your advice correctly, the GUI will appear in the HMD no? I'm looking to display a gui on the desktop mirror but not in the HMD. Is it possible ?

No, the GUI will never appear in the HMD unless you render the GUI to a separate viewport and use that viewport as a texture on a Quadmesh or something similar.

The steps I described here will present the GUI on the desktop, not on the HMD.

ultrasuperpingu commented 7 months ago

Ok, I tried this but I guess I didn't understand it well, I couldn't make it work :( I'm not sure I understand the logic. If I understand correctly, you suggest to render the entire scene in a subviewport (configured with use_xr=true), render it in a texture (with a TextureRect configured with a ViewportTexture pointing to the subviewport) and use this textureRect in a control on the main viewport (window) and then add the GUI on this viewport. Am I right ?

I tried that and it's ok in the HMD and the desktop mirror window display the gui correctly but the "left eye texture" is black. Texture is ok when I use a regular Camera3D and the subviewport is not "use_xr". Here is my setup: Godot_hierarchy_subviewport

But even if I make it work, it seems a little complicated and more than that, it's very intrusive just to display a GUI.

I understand this is like this probably by design and for good reasons. If it's the case, this issue should be closed (and maybe a proposal on this should be open). But if the GUI pipeline is avoid on the desktop viewport because 2D GUI does not make sense in the HMD, maybe it should be added back on the desktop mirror ?

BastiaanOlij commented 5 months ago

@ultrasuperpingu unfortunately that is a bug I recently ran into as well but haven't figured out why.

This worked fine as an approach in Godot 3, but it seems to not work specifically with OpenXR in Godot 4 (it does work with the native mobile interface), so there is something not correct with the way we do things here.

I am planning to investigate this soon.