Support for Lynx R1

[hammerandtongs]

The Lynx is beginning to ship to kickstarter backers and developers.

The sdk is currently only a 3 month old unity project but there was statements that godot was one they wanted to support.

https://portal.lynx-r.com/support/category/openxr

Currently these OpenXR extensions are listed -

XR_EXT_hand_tracking: Version=4 XR_KHR_android_create_instance: Version=3 XR_KHR_opengl_es_enable: Version=8 XR_ULTRALEAP_hand_tracking_forearm: Version=1

with support for

XR_KHR_vulkan_enable2: Version=2

still buggy in their Unity sdk.

[BastiaanOlij]

Sadly I don't have access to an R1 Lynx to verify, but if they are running a proper OpenXR runtime then the Khronos loader that we merged today should be enough to try and run Godot on their device.

[rodolpheh]

I have access to a Lynx R1 and tried the KHR loader (thanks for merging it, that just came in on point), but it doesn't seem to work. It goes further than my previous attempts (starts in immersive mode) but it seems to fail enumerating devices. See image and filtered logcat output attached.

I've also tried just in case to create a loader with the openxr_loader.so lib taken from Lynx's Unity SDK but I end up with the same results. As far as I can tell from the manifests it looks like it's using the KHR loader anyway (and their Unity doc indicates to choose the KHR Controller profile).

Let me know if you want me to try anything to help to get this to work.

filtered-logcat.txt

[hammerandtongs]

@rodolpheh

So I think that adding XR_LOADER_DEBUG=all to your godot android runtime and then reposting the logs would be helpful (I don't have a android xr device to confirm this).

[BastiaanOlij]

@rodolpheh ,

If you start from within Godot with remote debugging turned on in the menu you get info logged directly in Godot. You can also add this into your export settings to log far more info:

That said, looking at what you shared it's already failing on enumerating the extensions. That is one of the first things it does even before creating the OpenXR instance. That would suggest the loader isn't being loaded properly.

[rodolpheh]

@BastiaanOlij

I've added the --verbose extra arguments and used remote debugging but it doesn't seem to give more information regarding the issue with the loader.

The debugger gives this error:

E 0:00:00:0386   load_supported_extensions: OpenXR: Failed to enumerate number of extension properties
  <C++ Error>    Condition "((result) < 0)" is true. Returning: false
  <C++ Source>   modules/openxr/openxr_api.cpp:174 @ load_supported_extensions()

And the output shows:

--- DEVICE API >= 21; DEBUGGING OVER USB ---
Reverse result: 0
Godot Engine v4.0.beta17.official.c40020513 - https://godotengine.org
TextServer: Added interface "Dummy"
TextServer: Added interface "ICU / HarfBuzz / Graphite (Built-in)"
Using "default" pen tablet driver...
godot: ENABLING GL DEBUG
OpenGL Renderer: Adreno (TM) 650

TextServer: Primary interface set to: "ICU / HarfBuzz / Graphite (Built-in)".
CORE API HASH: 1230509124
EDITOR API HASH: 403565994
Loaded builtin certs
OpenXR not initialised, please check if your headset is connected

@hammerandtongs I'm unsure how I can add this environment variable to the application. Android is a strange beast.

[kisg]

Hi @rodolpheh,

I see that you are using the official build. When trying to get a new device running, it is best to build the engine and plugins yourself in debug mode. It might also make sense to add more debug output to the place where the above error is returned (it is in openxr_api.cpp, line 173-174 on current master.), so we know the exact error code returned by xrEnumerateInstanceExtensionProperties.

You can set an environment variable on Android using the setenv() function. A good place for that would be here, around line 1200 in openxr_api.cpp, before the open_dynamic_library() call:

bool OpenXRAPI::openxr_loader_init() {
#ifdef ANDROID_ENABLED
    ERR_FAIL_COND_V_MSG(openxr_loader_library_handle != nullptr, false, "OpenXR Loader library is already loaded.");

    {

This will make sure that when the Lynx OpenXR Loader is initialized, the environment variable is already set.

This Lynx R1 seems like a great device, would love to get my hands on it - and add Godot support to it of course (we implemented the current OpenXR loader support for Godot on Android).

[BastiaanOlij]

@kisg in theory it should already work hence the testing here. Ron Bessems added support for the standard Khronos loader in the loader plugin. But it's only been tested on the Magic Leap.

Stan Larroque expressed interest in seeing it work so fingers crossed they will have a look into things.

I suspect it's something simple that is preventing the loader from being loaded, indeed adding some more logging to the part where we're loading stuff might reveal the problem.

[rodolpheh]

Thanks everyone, I appreciate the help. I did a build of the latest Godot version from master and added a couple of ifs and prints to debug what error was coming out of the enumeration and what I get is XR_ERROR_RUNTIME_UNAVAILABLE.

[kisg]

@rodolpheh Are you using the KHR loader or the loader provided by Lynx in their Unity SDK?

[rodolpheh]

I'm using the KHR Loader. I've tried using their in a custom loader copied from the KHR loader, but it doesn't seem to work either. I'll retry with their loader, just replacing the KHR .so file with theirs.

[rodolpheh]

I have just switched the openxr_loader.so in KHR loader for theirs (and rebuild the aar) and tried again but I get the same error.

[kisg]

Ok, the next step would be to check in the KHR loader source code how you can get this specific error. My hunch is that it cannot find the actual OpenXR runtime implementation. You could also try setting XR_LOADER_DEBUG=all environment variable suggested above, to see if that provides more insight into what the loader tries to do. You could also check the KHR loader source / add more debug statements if necessary.

You could also check the official sample app to see if that works. If that works with their loader, then you could try to swap out that loader for the KHR loader. If that works, then you should be able to see what is different in the Unity environment compared to Godot.

[BastiaanOlij]

Could it be that the Lynx R1 requires extra entries in the manifest to tell it the app is a proper XR app? We had a similar issue on the HTC

[kisg]

Good point @BastiaanOlij, @rodolpheh you could check the Android manifest configured in the Unity sample app.

[rodolpheh]

Well I have some good news ! So previously I had access to an application for the Lynx and used apktool to check the Manifest. There were a few elements missing in mine that I added but it didn't work. I just downloaded the sample app given by Lynx and disassembled this one as well. I found a small snippet in the Manifest that I didn't have in mine. Adding it did the trick ! I even have hand tracking working (although not displaying properly yet in my scene, will work on that) !

Here is the addition that made it working:

    <uses-feature android:name="android.hardware.vr.headtracking" android:required="true" android:version="1"/>
    <uses-permission android:name="org.khronos.openxr.permission.OPENXR"/>

    <queries>
        <intent>
            <action android:name="org.khronos.openxr.OpenXRRuntimeService"/>
        </intent>
        <provider android:authorities="org.khronos.openxr.runtime_broker;org.khronos.openxr.system_runtime_broker" />
        <package android:name="com.ultraleap.tracking.service"/>
        <package android:name="com.ultraleap.openxr.api_layer"/>
    </queries>

I also added <meta-data android:name="handtracking" android:value="1" /> in the activity part.

I will check a few more things to make sure my scene is displayed and work properly. Let me know if there is anything I can do to help improving upstream.

EDIT: just for the precision, the part that really did the trick was the intent part.

[BastiaanOlij]

@rodolpheh was this testing with beta 17 and with the Alpha 4 version of the loaders plugin? I'm thought we had added the Khronos entries in those builds, they indeed weren't in older ones.

[rodolpheh]

This was tested with Godot built from the main branch from yesterday and the openxr_loaders were built from the master branch (from yesterday as well). I've been switching a lot of versions around, I'll make some cleanup and try again tomorrow to make sure I can reproduce it.

I've checked the latest Godot source and can't find any reference to the OpenXRRuntimeService intent. Could this be the missing piece ?

EDIT: I was also using Lynx's openxr loader lib. I'll double check to see if this also helped.

[BastiaanOlij]

@rbessems @m4gr3d could you comment on the above please? I thought we added everything required for the KHR loader but it does seem org.khronos.openxr.OpenXRRuntimeService may be another requirement here that may need to be added to core?

[kisg]

I think these changes to the manifest can be added to the KHR loader .aar manifest here, and the Android Gradle build system will merge them into the app manifest automatically. There should be no need to add these to the core.

[rbessems]

org.khronos.openxr.OpenXRRuntimeService is documented here: https://registry.khronos.org/OpenXR/specs/1.0/loader.html

[rbessems]

@BastiaanOlij below looks like a good addition to KHRLoader extension. I'd leave the vendor specific stuff out.

    <queries>
        <intent>
            <action android:name="org.khronos.openxr.OpenXRRuntimeService"/>
        </intent>
        <provider android:authorities="org.khronos.openxr.runtime_broker;org.khronos.openxr.system_runtime_broker" />
    </queries>

[hammerandtongs]

@rodolpheh I added #26 with a possible method of exposing XR_LOADER_DEBUG=all on andoid

I think this would be useful for future porting efforts to find out if this works and is useful (I can't test it).

[BastiaanOlij]

@BastiaanOlij below looks like a good addition to KHRLoader extension. I'd leave the vendor specific stuff out.

    <queries>
        <intent>
            <action android:name="org.khronos.openxr.OpenXRRuntimeService"/>
        </intent>
        <provider android:authorities="org.khronos.openxr.runtime_broker;org.khronos.openxr.system_runtime_broker" />
    </queries>

I'd be happy to take a PR for that, I'm a little hesitant to do this myself without the ability to test it (which, fingers crossed, might be resolved later today...)

[rodolpheh]

I'll be traveling for work today so won't be able to have a look until monday. Sorry everyone. This is a great conversation and I love how fast this is going :heart:

There is a conversation that needs to happen about vendor specific stuff, specially regarding hand tracking (the settings in Godot are only for Meta/Oculus) but that's outside of the scope of this thread. I'd be happy to help testing on that as well.

[rodolpheh]

I've made multiple checks and found out that it works only when using the OpenXR loader lib from Lynx instead of the KHR one, so it seems that it is different from the KHR one. I'll try to take some time to create a specific loader and make a PR.

[m4gr3d]

@BastiaanOlij below looks like a good addition to KHRLoader extension. I'd leave the vendor specific stuff out.

    <queries>
        <intent>
            <action android:name="org.khronos.openxr.OpenXRRuntimeService"/>
        </intent>
        <provider android:authorities="org.khronos.openxr.runtime_broker;org.khronos.openxr.system_runtime_broker" />
    </queries>

I'd be happy to take a PR for that, I'm a little hesitant to do this myself without the ability to test it (which, fingers crossed, might be resolved later today...)

I can take a look adding that once @rodolpheh's PR lands. @rbessems Were you able to confirm the addition of the queries tag in the manifest doesn't affect Magic Leap's support?

[rbessems]

@m4gr3d I'm currently at OpenXR conference without the hardware. Will test on Friday, but I don't expect trouble with that addition.