OpenXR: Controller tracking fails sometimes when `pose` is set to `default`

jtnicholl commented 1 year ago

Godot version

v4.0.beta.custom_build [1bfaa73e1]

System information

Fedora 36, SteamVR beta 1.25.1, HTC Vive

Issue description

When launching certain VR projects, XRController nodes don't move. There are no errors printed. I can't figure out what causes this. The MRP attached triggers it for me, the controllers won't move. A project I was working on was tracking fine until I edited the action map, then it stopped tracking. Resetting to the default action map and clearing .godot/ didn't fix it. The project at https://github.com/BastiaanOlij/godot4_openxr_demo does track for me, consistently every time I run it. In summary, it seems mostly random. I know this probably isn't the most helpful of bug reports, but I'm at a loss here. It may be related to the action map, which would mean this is something similar to #65395, but I have no idea. It might also be OS specific, VR on Linux isn't too popular.

Steps to reproduce

Open certain VR projects

Minimal reproduction project

xr_controllers.zip

BastiaanOlij commented 1 year ago

The project is working for me on Windows with an Index.

Crazy suggestion is to change the anti aliassing setting to 2x, there is a known issue I'm still chasing where the depth buffer breaks in VR and it may be tracking your controllers but not rendering the spheres correctly.

jtnicholl commented 1 year ago

That didn't make a difference. The spheres are rendering and I can move the camera just fine, but the spheres don't move. However, when going back to try it, the controllers did start tracking after I opened and closed the SteamVR dashboard. It only happened once, when I tried again I couldn't replicate it. Makes me think this might be a SteamVR issue, though if that is the case, I don't know why it happens only with certain Godot 4 projects.

BastiaanOlij commented 1 year ago

One thing that may be related is that controllers often are "paused" when you the proximity sensor isn't triggered or if a steamVR menu is open. That is by design in OpenXR. Then again, if you are wearing the headset you should see the tracking working.

One other thing to try is start godot with --verbose, it does a lot more output that may help

ModularNucleus commented 1 year ago

I have had this happen too.

I can still make it happen, although I'm not quite sure whether this is what happened in your case, nor whether it can be considered an actual bug:

If I remove the "aim_pose" action from the OpenXR Action Map, my HPMR controllers (naturally?) stop tracking. This happens even though the HPMR profile shows the Aim Pose as still bound to the "default_pose" action (as I believe the actions are the same, internally?).
The controllers do not start tracking even if I add back the "aim_pose" action - written exactly as originally - and press save.
Only when I also rebind the Aim Pose on the HPMR profile to the readded "aim_pose" action or the "default_pose" action, do they start tracking again.

I guess this could happen by accident and create a bit of confusion, since the profile page still shows an action bound and there seems to be no indication that the binding is invalid?

BastiaanOlij commented 1 year ago

Ok, the "default_pose", "aim_pose" and "grip_pose" are three separate actions, the default pose was added purely for backwards compatibility but they don't have any ingrained meaning.

You could remove them completely and create your own poses if you want, it's just a name with which Godot recognizes that you need access to a pose. The action map we supply is just a default, it can be tossed away and started from scratch if you choose to do so.

In the end it is the action you select on the XRController3D node that defines which action Godot queries when positioning that controller.

If you do not map that action to the actual input pose on the HPMR profile, then yes, that pose isn't functional. OpenXR does not know which input to feed into the action.

With OpenXR everything is done through named actions. Poses, buttons, triggers, joysticks, haptics, everything is identified by a named action that is bound to the appropriate input or output on the controller. The idea being that as a game developer, you define a number of actions with which you create your game without worrying about how those actions are bound.

With poses it's a little silly, all controllers have an aim and grip pose as inputs, so making the binding configurable seems a little over the top but when you think of it as a complete system, and you see how different controller can have different inputs (some have a trackpad, others a joystick, some have both, some have a menu button, some have A/B/X/Y buttons, some future controller may have something we haven't seen before) the mapping system makes a lot of sense.

The idea here also is that the mappings you do supply are suggested mappings to the XR runtime. The XR runtime will use the mapping for the controller the user has if available, also it will try and find the best mapping one. How is up to the runtime to decide.

It's also the idea that you only supply bindings for controllers you have access to, because indeed you as a developer are responsible for making sure all the required bindings are made. If you don't bind an action, OpenXR just assumes that action is not supported by that controller and you as a developer have made sure that you deal with that scenario.

BastiaanOlij commented 1 year ago

Note that there is a lot of overlap between Godot input system and OpenXRs action system, so think of it the same way.

In Godot input system you create an action called "Fire", then you bind a key to it for keyboard users, and a button to it for gamepad users. If no input is bound for your device, you can't trigger "Fire"

OpenXR is similar, except for a couple of key differences:

actions are typed, you have to specify if an action is a pose, bool, float (i.e. trigger), vector2 (i.e. joystick or trackpad) or haptic (i.e. output), OpenXR will convert between bool and float.
you make bindings for every type of controller
you can't change the bindings during runtime (though there is a convoluted way but we've not implemented it (yet) because it is riddled with holes)

If it wasn't for these differences we would probably have integrated the action map into Godots input map, that was our original plan but the devil is in the details :)

jtnicholl commented 1 year ago

Note that there is a lot of overlap between Godot input system and OpenXRs action system, so think of it the same way.

In Godot input system you create an action called "Fire", then you bind a key to it for keyboard users, and a button to it for gamepad users. If no input is bound for your device, you can't trigger "Fire"

I think the default action map is confusing some people because it isn't set up that way. Instead of actions like "fire", "teleport", or "grab", it has actions like "trigger", "trackpad click", etc.

BastiaanOlij commented 1 year ago

Indeed, but that is something that will need to become a tutorial. The actions you'd want are different depending on the game you're developing.

This map is purely designed to make porting VR games in Godot 3 to 4 "easier".

jtnicholl commented 1 year ago

Ok, I've found something out. This only happens if the pose property on the controller is set to default. That explains why the demo project I linked always works, the pose on the controllers is set to aim there. So I've found a way around this for now, at least. Still something odd going on.

BastiaanOlij commented 1 year ago

Sounds like default isn't bound on all controllers in your action map. I wonder if we should get rid of it all together.

HollisRules commented 1 year ago

Okay so I had a similar issue. When I removed the default action set the controllers would no longer track. So in my action set I made a new Pose action name pose and applied it to the pose I was using but that didn't work either. What ended up working was copying exactly how the default action set defines the poses and applying them exactly the same way to the profile. So default_pose aim_pose grip_pose palm_pose worked for me. I don't know what deeper reasons caused that but it worked immediately after I copied it exactly

godotengine / godot