Add support for the Oculus platform keyboard

[m4gr3d]

See https://developer.oculus.com/documentation/unity/unity-keyboard-overlay for reference.

[jakobbouchard]

Hey! I’m currently building an app using Godot, targeting the Quest headsets and was wondering, if there’s no support for the keyboard, how would I get text input in VR? Do I need to implement my own keyboard?

Also, from what I’m understanding from the other issues, Oculus is deprecating their API in favour of OpenXR. How will this affect this plugin?

[BenMcLean]

Do I need to implement my own keyboard?

AFAIK, yes. Personally, I'm having to flat out design my app differently to avoid text input because of this.

[m4gr3d]

@jakobbouchard As you mention, due to the deprecation of Oculus vrapi in favor of OpenXR, this plugin will be shortly deprecated. We'll be updating the repo README to reflect that state.

As such, we're planning to add support for the Oculus platform keyboard in the https://github.com/GodotVR/godot_openxr plugin instead (see https://github.com/GodotVR/godot_openxr/issues/136).

The OpenXR plugin will be the default Godot plugin used to support VR on Quest (and other VR headsets), so we recommend that you start migrating to it as soon as possible.

Let us know if you have any questions!

[jakobbouchard]

Are most Quest-specific features already supported in the OpenXR plugin? Hand tracking for example?

[m4gr3d]

Yes they are! We have one remaining issue with color correction to address, but asides from that, the plugin is ready for use.

The version to start testing against should be 1.1.1.

[jakobbouchard]

I don't see version 1.1.1 in the releases, only 1.1.0. Not sure how to build it myself, but I might try. Unless the latest automatic build 780fbbb is good to go?

[m4gr3d]

@jakobbouchard There are a couple remaining features before we release it as stable. Prior to this version, hand tracking is broken on the Quest, hence why I'm recommending it as a starting point.

The latest automatic build should have the necessary prebuilt libraries. Once you have them, you can follow these instructions.

[m4gr3d]

cc @BastiaanOlij

[BastiaanOlij]

I don't think there is virtual keyboard support in the OpenXR spec, this sounds like a platform dependent feature that should be added to the Oculus platform plugin?

Also creating a virtual keyboard in Godot itself shouldn't be a hard thing to do and then you have a solution that is portable between multiple platforms, something that would fit well in a toolkit such as xr-tools or neosparks toolkit.

[m4gr3d]

They are working on making it available via Openxr. It'll only be integrated into the plugin once that's done.

And actually a full keyboard implementation is very complex to get right. On top of the basic UI and interaction scheme to figure out (e.g: single/multi controller/hands interaction), you also need to support all the locales supported by the platform (e.g: European languages, asian languages, rtl languages, etc).

[BastiaanOlij]

@m4gr3d Never said it was easy, looking at the immense work that has gone into Godot to support Arabic and eastern languages, I don't even know where to start designing a keyboard implementation that supports this (I would stick to what I know and create a QWERTY based virtual keyboard and hope contributors step up and build ontop of that).

It's just that I think a keyboard implementation should be done in the game engine. first because the game engine has the best tooling available for this, second because now every XR runtime is going to have to implement their own version and they are going to behave differently on each platform, and some may not have it resulting in us having to have a keyboard fallback in the game engine anyway.

That said, as so often in these types of discussions, we're looking at it from the perspective of having a full fledged game engine with 2D rendering support with existing user made libraries for virtual keyboards as this is a requirement for console gaming as well. It's an entirely different story for someone scratch building a VR game on top of pure OpenXR.

[BenMcLean]

now every XR runtime is going to have to implement their own version and they are going to behave differently on each platform

I wish Godot had the power to tell the OpenXR-compatible runtimes, "No, you will implement the spec." But unfortunately, you are right. It'll have to be in the game engine.