search

godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.15k stars 97 forks source link

Make the default OpenXR action map more semantic #5951

Open rsjtdrjgfuzkfg opened 1 year ago

rsjtdrjgfuzkfg commented 1 year ago

Describe the project you are working on

Toy projects to get a feeling for virtual reality with Godot.

Describe the problem or limitation you are having in your project

The current (Godot 4.0 / trunk) default action map behaves in unexpected ways when switching between headsets / input devices, as it is focussed on concrete buttons and not semantic features. Furthermore, it does not follow the recommended way to set up action maps in real-world projects.

This can lead to input actions not being available when switching headsets, as well as developers making suboptimal decisions by following a bad example when setting up their own action maps based on the defaults.

Having a better default would make the "right way" more discoverable and ensure that demo projects that do not need to change the action map run on most hardware in the best possible way.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The default action map should be changed to provide a basic set of actions across all input devices, based on the capabilities of khr/simple_controller. All actions of this basic set should be available and easily accessible independently of which controller is actually used.

Furthermore, we should reconsider other actions to provide a consistent experience across controllers.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Minimal change: map select and menu on every controller in an easily accessible way. The obvious option to do that would be to add select to clicking the trigger on most controllers or to map trigger_click to select in khr/simple_controller. menu is already mapped everywhere, though I don't have all controllers and thus cannot warrant it is always accessible.

Preferred solution: we pick a new set of semantic actions as default action set, instead of referencing specific buttons / input mechanisms. Then map them for each controller in the most reasonable way. This could cause compatibility issues with 3.x, but would greatly increase discoverability of the "right" way to do actions in Godot (as the default action map currently violates the design principles we recommend to developers).

Possible new default action set, basic actions (available everywhere) in bold:

If this enhancement will not be used often, can it be worked around with a few lines of script?

This enhancement affects default values for new projects, so it would be used in every single OpenXR project until a developer invests time in adapting the action map to the individual game's needs. Given the nature of default values, it can be worked around by changing the configuration in every single project.

The current defaults are a bad example not following the OpenXR guidelines and causing trouble for developers that are not familiar with OpenXR. We should thus change the defaults to make Godot more accessible.

Is there a reason why this should be core and not an add-on in the asset library?

The default values to change are in core, so the change needs to happen in core.

(It is already easy to change the defaults if a developer is aware that the defaults are not ideal for them, without add-ons or assets.)

rsjtdrjgfuzkfg commented 1 year ago

Pinging our resident OpenXR expert and stakeholder @BastiaanOlij

Two main questions to discuss imho:

  1. Do we agree that making something akin to select / trigger_click available on all controllers is something we want to do?
  2. Is changing the default actions more drastically something we can do from a compatibility standpoint? If so, do we agree on the proposed new semantic action set?
jtnicholl commented 1 year ago

I mentioned this fact in https://github.com/godotengine/godot/issues/69937#issuecomment-1350376645.

I was told it's set up this way to make porting projects from 3 to 4 easier. The OpenXR addon for Godot 3 doesn't support proper action binding, and the default is emulating it. Porting to Godot 4 is never going to be effortless, I don't think making decisions like this for backwards compatibility with what was technically a third-party addon is a good idea.

2. Is changing the default actions more drastically something we can do from a compatibility standpoint? If so, do we agree on the proposed new semantic action set?

The default action map gets created if there isn't already one. Changing it has no effect on existing projects, it won't affect compatibility.

Since actions should be specific to the game/app being made, I think it would be better to have the default be very minimal, or even completely empty.

BastiaanOlij commented 1 year ago

As I've mentioned in other threads, I'm on the fence with how we should proceed forward.

The "official" stance of the OpenXR specification is that you create actions based on your games features ("Fire", "Menu select", "Move", etc.) and then map those actions to the interaction profiles for controllers you test on. It is then up to the XR runtime to fall back on automatic logic if no interaction profile can be found for the hardware used by the user.

Now that last one is a bit of an issue, as @rsjtdrjgfuzkfg hints at, khr/simple_controller is very limited and for most modern VR games does not contain enough inputs while almost every controller on the market does have the inputs we're used to using. Valve as a result first falls back on the touch controller, Microsoft (not that we currently support native WMR) as far as I'm aware has multiple fall backs and will have WMR controllers emulate a variety of other controllers. I'm not sure what Meta does but their hardware is currently so prevalent they can probably assume a touch profile is present.

Alternative runtimes, such as the PICO runtime which set off this discussion, still fall back on the simple controller giving a subpar experience. There is a whole other story to that that I won't get into here.

So on face value, the fact that we have an action map that almost 1:1 maps inputs to actions and maps all known profiles whether we have access to test with the relevant hardware or not, goes against the principles of OpenXR and will set developers on the wrong foot.

On the other hand, if we make a very basic action map with just a few default inputs, and only one or two of the core interaction profiles, we force developers to implement their action map first before things "work out of the box", which is a terrible user experience. I have to admit I'm more and more leaning to going down this path provided we first write some really good documentation around setting up an action map (something that is on my list of todos). For people converting from Godot 3 to Godot 4 we can always make a fully fledged action map resource file downloadable.

Last but not least, the OpenXR WG is aware of some of these pitfalls, we're far from the only game engine running into difficulties here, so we also need to keep in mind that the OpenXR spec is still fairly young and will do some growing up. While we can't make any assumptions on what these improvements will be, some of these issues will likely be resolved in due time.

Same can be said for us, not many people are really experimenting with the action map yet, and those who've worked in Godot 3 whose input system was designed before we had action maps, still have to figure out the best practices here (not to mention help find and resolve the bugs in the action map interface).

BastiaanOlij commented 1 year ago

@rsjtdrjgfuzkfg just a foot note btw, menu is a PITA because they're often seen as system buttons and already have functionality assigned to them by the host system (menu in SteamVR will open the SteamVR menu for instance). Thus while exposed in the input bindings, often they are not usable.

rsjtdrjgfuzkfg commented 1 year ago

@jtnicholl @BastiaanOlij you raise some good points about limiting the feature set, though I would not want to start with an empty set. I did remove the secondary direction input in my proposal above to somewhat simplify things as I too felt the current default does too much, but we could go even further.

That said, imho it would be ideal if quick and dirty test projects do not need to adjust the action map. That would permit developers to get running quickly, especially if they just want to try something out real quick. A balance is needed.

I still think the proposal at the top is reasonable to achieve this goal, though I'm open to cutting it down up to the base feature set (all actions that are marked as bold) and then only supplying a mapping for khr/simple_controller. I would not like the default to be less than that, as that makes quickly trying things much more cumbersome.

Regarding menu: I think having one action that can be used for "get me to a meta layer" (most commonly a menu) is useful for many games and test projects. We should definitively map it to a button that is not used by the system, though. I thought that is already the default (Pico has a dedicated "menu" key that is not used by the system and only for games, in addition to three system keys usually controlling the system [with some remapping support, but not via OpenXR]).

BastiaanOlij commented 1 year ago

@rsjtdrjgfuzkfg I haven't really made up my mind on this to be honest, your arguments here make total sense to me, at the same time I keep falling back to the same design paradigm around Godots input map where only actions are added that are needed by Godots UI controls. Every tutorial starts by "add these actions to the input map" and the actions being added are very different from one game to another.

But that leaves a huge gap for people who just want to hit the ground running, not to mention that people who don't read the manual, may quit trying to make things work before they figure out they have to setup an action map.

So yeah, there are pro's and con's to both scenarios and I'm unsure which one wins out :)

As for the menu button, Oculus controllers for instance have it so that the button on the right controller is the system one that you can't access, it opens the Oculus menus, then on the left controller the same button you can use for your in game menu. I think Index is similar but I don't know which is which as they is no logo on either button :)

Also one thing we under utilise is action sets. There is actually an outstanding task for me to allow you to enable/disable action sets on the OpenXR interface.

The way OpenXR wants you to implement this is to create two action sets. Lets call the first action set "Game" and this has your "Fire", "Movement", "Open Menu", "Aim Pose", etc actions, the stuff that makes your game work. The second one is called "Menu" and only has a "Select" and "Aim Pose" action (this last one actually should have a different name, it is not strictly enforced that action names should be unique but in practice having two actions called the same, even if they are in different sets, causes all sorts of issues). The Game action set is active and the Menu action set is inactive when you're playing your game. Then when the user presses the button mapped to "Open Menu" you pause your game, disable the Game action set, enable the menu action set and show your menu. This ensures you can have a dedicated set of actions for interacting with your menu, while disabling other actions. You can have multiple sets active at the same time so moving actions that are always required (like "Aim Pose") into a separate action set can be a good organizational choice as well.

But again, that is assuming that this is the way you want to build your game. There are many VR developers that believe menus are immersion breaking and swear by in world menus, either by having it on a wrist device or otherwise. Does it make sense to have such an setup in a default action set, or should this be relegated to tutorials/documentation. In this particular case I strongly lean to the later.

rsjtdrjgfuzkfg commented 1 year ago

@BastiaanOlij I agree on action sets being something we should leave to developers. Our default should be simple.

Let's reduce the discussion so far into concrete options to make a decision (I hope I captured everything?!):

Option 1: remain flexible, but more semantic This is the original proposal from the first post: default to a single action group with the following actions, mapped for each controller as far as reasonably possible:

For me personally, this feels like the best option. At least for me it is common to start a new project to quickly try something, and not having to configure anything for such tests would feel best.

Option 2: limit to basic features in a semantic set This is a subset of Option 2, aiming to drastically reduce the amount of features while keeping inputs of all types (float, bool and Vector2) and basic operations (context-related trigger, generic action and something to go back / to a menu). These actions would still need to be mapped for multiple controllers to be useful.

From the past discussion here, this feels like a good compromise to improve the discoverability of action maps ("how can I use X?" might trigger people to search for it and find action maps in the first place) while still providing sufficient defaults for most tests and toy projects.

Option 3: limit to simple_controller The default will contain a single action set, which is only mapped for the simple controller. This forces developers to make a more proper action map early on, while still having some defaults for simple tests. However, not all engine features can be used out of the box, due to the lack of Vector2 or float inputs.

For me personally, this feels limited, but still better than not providing a default at all.

Option 4: empty Having no real default, but instead starting with an empty action map would ensure that all developers necessarily have to think about action maps. On the other side, this would be most inconvenient for toy projects and short tests.

While I would prefer to start empty over the current default, not having any default can be quite inconvenient. I also feel it is not a good fit for the overall feeling of Godot when prototyping: many existing features aim to be at least somewhat useable with minimal effort.

Option 5: leave as-is I feel we all agree that not changing anything is not a good idea, but it is an option we can take – and will take by default if we cannot make a decision here.

Imho this is the worst option, as it suggests non-semantic naming conventions and can cause inconsistent behaviour across controllers.

As stated, I personally prefer Option 1 but feel Option 2 might be the best match for Godot given the arguments and use cases presented. I hope we can avoid Option 5, but it will be selected automatically if we cannot make a decision before Godot 4 becomes stable. So I'd like to see us make a decision soon ;)

Not sure what the official process is to proceed / make an official decision, though. @BastiaanOlij ?