Issues found on one APK handling both 3dof and 6dof

[lzhangcs]

Per HVR's feedback this Monday: Huawei VR SDK can support One APK for both 3dof and 6dof. But App side needs to do the following: 1、Android manifest.xlm : • <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/>

App needs to check in real-time to see which one is connected, 3dof or 6dof ? and then handle the handler's type accordingly. . Under openxr, use "XrSpaceLocationFlags" to check if it is 3dof or 6dof . if it is 3dof, it only has Orientation; for 6dof, it has both Orientation and Position.

After tried out above suggestions, CP found the following issues:

1. Regarding 2, we do indeed use those OpenXR flags to figure out whether we have positional tracking or not. However we don not create the OpenXR session until we enter the immersive mode of the browser. What we would like is some system/native API to detect the type of controller earlier, when the application is starting (i.e. in the Java code).

2. There is a bug in the OpenXR SDK that was reported some time ago related to 3dof/6dof detection.

Using XrSpaceLocationFlags is possible but not ideal because that's a property of a XrSpace not a property of the XrSystem. We could use that (for some cases) but it is not the most correct way of checking it. The proper way of detecting that is by calling xrGetSystemProperties() and then check the systemProperties.trackingProperties.positionTracking value. The problem is that the HVR SDK always returns false for that, meaning that it always claim to be a 3DoF device.

3. So I tried the suggestion of the HVR team and I can confirm that it does not work. I checked the values of XR_SPACE_LOCATION_POSITION_VALID_BIT, XR_SPACE_LOCATION_ORIENTATION_VALID_BIT, XR_SPACE_LOCATION_POSITION_TRACKED_BIT, XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT and I can safely conclude that:

• If the AndroidManifest.xml declares that the app is 3dof then we indeed get only we orientation bits
• If the AndroidManifest.xml declares that the app is 6dof then we get both position and orientation
• If the AndroidManifest.xml declares that the app is 3dof|6dof then we get both as well

This does not depend on having the 6DoF kit in the glasses. I verified that only depends on what you declare on the manifest. This means that if you declare that your app is 6DoF and you don't have the 6DoF kit, the SDK will still tell you that you have positional tracking. That behavior seems completely wrong. The bits should depend on whether you have the 6DoF kit or not, not what you declare in the manifest.

So even with the suggestion from the SDK team, there is no way to check that at runtime. Again we need the SDK team to fix this issue so that we know for sure whether we have 6DoF or 3DoF controllers at runtime.So I tried the suggestion of the HVR team and I can confirm that it does not work. I checked the values of XR_SPACE_LOCATION_POSITION_VALID_BIT, XR_SPACE_LOCATION_ORIENTATION_VALID_BIT, XR_SPACE_LOCATION_POSITION_TRACKED_BIT, XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT and I can safely conclude that:

If the AndroidManifest.xml declares that the app is 3dof then we indeed get only we orientation bits If the AndroidManifest.xml declares that the app is 6dof then we get both position and orientation If the AndroidManifest.xml declares that the app is 3dof|6dof then we get both as well

This does not depend on having the 6DoF kit in the glasses. I verified that only depends on what you declare on the manifest. This means that if you declare that your app is 6DoF and you don't have the 6DoF kit, the SDK will still tell you that you have positional tracking. That behavior seems completely wrong. The bits should depend on whether you have the 6DoF kit or not, not what you declare in the manifest. So even with the suggestion from the SDK team, there is no way to check that at runtime. Again we need the SDK team to fix this issue so that we know for sure whether we have 6DoF or 3DoF controllers at runtime.

4. There is another very important issue that might block having an unified build. It's this issue which I already reported. The problem is that with a unified build we need to register both 6DoF and 3DoF input profiles (in the past we only registered one on each build). If we do that then the SDK SIGSEVs (i.e. crashes) in a follow up call. We could add a workaround for that 😞 but we need a solution for [what I mentioned above ] first.

[lzhangcs]

@lixming Please take a look of above issues that CP found. Thanks

[svillar]

Just to clarify, because the above comment is a mix of 3 different issues:

1. 3dof/6dof should be checked using xrGetSystemProperties() and then check the systemProperties.trackingProperties.positionTracking. Right now it always returns 0, so it is not usable. This is issue #35
2. The position/orientation values set in XrSpaceLocationFlags that can be queried with xrLocateSpace() are not correct because they only depend on what the application declares in the AndroidManifest.xml (even if you don't have the 6DoF kit the runtime returns POSITION_TRACKED as long as the manifest declares that the app is 6dof). The runtime must also check the presence of the 6DoF kit and act accordingly.
3. It is not possible to register multiple interaction profiles. We need that if we want to have a hybrid 3dof/6dof application. We cannot do that because the OpenXR runtime crashes, that is issue #34

[Lixming]

It seems it did exist some bugs about the systemProperties.trackingProperties.positionTracking and XrSpaceLocationFlags in the situation you metioned above. Just try to set <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/> and register 6Dof profile. This can ensure that your application can be used in 3/6 DOF vrglass, and can correctly obtain the buttons info and pose of the controller.

[svillar]

It seems it did exist some bugs about the systemProperties.trackingProperties.positionTracking and XrSpaceLocationFlags in the situation you metioned above. Just try to set <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/> and register 6Dof profile. This can ensure that your application can be used in 3/6 DOF vrglass, and can correctly obtain the buttons info and pose of the controller.

The problem with that is that we wouldn't be able to detect whether the user is using the 3DoF or 6DoF controllers unless the issues I mentioned are fixed.

[dingsing2021]

It seems it did exist some bugs about the systemProperties.trackingProperties.positionTracking and XrSpaceLocationFlags in the situation you metioned above. Just try to set <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/> and register 6Dof profile. This can ensure that your application can be used in 3/6 DOF vrglass, and can correctly obtain the buttons info and pose of the controller.

The problem with that is that we wouldn't be able to detect whether the user is using the 3DoF or 6DoF controllers unless the issues I mentioned are fixed.

can you bulid a apk for the next steps? we can take a experience of it.

<meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/> and register 6Dof profile.

[dingsing2021]

it seems simple to bulid a new apk for test, just from the already existing bulid project of 6dof version, modified the manifest.xml , then build it. <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/>

[svillar]

it seems simple to bulid a new apk for test, just from the already existing bulid project of 6dof version, modified the manifest.xml , then build it. <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/>

Yes, building is not a problem at all. What I mean, is that after setting 3dof|6dof in the manifest file, the OpenXR SDK will start telling the applications that the controllers are 6DoF. This won't be a problem for 6DoF controllers but will break the experience with 3DoF controller. Please review all the issues I found and described in this comment

In any case, you requested a demo apk and I can give you one. Can be downloaded here.

[Lixming]

For the tempory usage, if you want to confirm the current controller type, you can judge whether the position value of the controller is (0, 0, 0). Although this judgment will be late, it is the compromise method provided by the current SDK. Binding 6dof profile can ensure that the controller infos can be obtained in both cases.

[dingsing2021]

it seems simple to bulid a new apk for test, just from the already existing bulid project of 6dof version, modified the manifest.xml , then build it. <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/>

Yes, building is not a problem at all. What I mean, is that after setting 3dof|6dof in the manifest file, the OpenXR SDK will start telling the applications that the controllers are 6DoF. This won't be a problem for 6DoF controllers but will break the experience with 3DoF controller. Please review all the issues I found and described in this comment

In any case, you requested a demo apk and I can give you one. Can be downloaded here.

thanks very much , I have took experience with 3dof controller,the HVR runtime maps the funtions between 6dof controller@right and 3dof controller,most of the fuctions run well, the triger,home,and volum button run good, the tacking of the 3dof controller and HMD runs good. but there is some experience problem, 1，controller model, The 3dof handle model is not shown, instead of two cursors are shown, one of which is completely stationary and the other that can moved with the 3dof controller. 2, no raycast 3, 3dof controller touchpad ,It doesn't seem to work logically. in the home page , use the touchpad to scroll the page up and down , but when you finger leave the touchpad , the page return the orginal, so I can't use the 3dof touchpad to scroll the page up and down . 4, the "return" button, it does not work , because wolvic use the left controller of 6dof, but in this mode, the runtime just maps the 3dof cotroller with the right 6dof controller.

[svillar]

it seems simple to bulid a new apk for test, just from the already existing bulid project of 6dof version, modified the manifest.xml , then build it. <meta-data android:name="com.huawei.vr.application.freeDegree" android:value=“3dof|6dof"/>

Yes, building is not a problem at all. What I mean, is that after setting 3dof|6dof in the manifest file, the OpenXR SDK will start telling the applications that the controllers are 6DoF. This won't be a problem for 6DoF controllers but will break the experience with 3DoF controller. Please review all the issues I found and described in this comment In any case, you requested a demo apk and I can give you one. Can be downloaded here.

thanks very much , I have took experience with 3dof controller,the HVR runtime maps the funtions between 6dof controller@right and 3dof controller,most of the fuctions run well, the triger,home,and volum button run good, the tacking of the 3dof controller and HMD runs good. but there is some experience problem, 1，controller model, The 3dof handle model is not shown, instead of two cursors are shown, one of which is completely stationary and the other that can moved with the 3dof controller. 2, no raycast 3, 3dof controller touchpad ,It doesn't seem to work logically. in the home page , use the touchpad to scroll the page up and down , but when you finger leave the touchpad , the page return the orginal, so I can't use the 3dof touchpad to scroll the page up and down . 4, the "return" button, it does not work , because wolvic use the left controller of 6dof, but in this mode, the runtime just maps the 3dof cotroller with the right 6dof controller.

I know, that's why I explained several times that we need those fixes in the SDK so we could detect whether we have a 3DoF or a 6DoF controllers because we need to load the proper models, we need to load the proper mappings, we need to set the proper transformations to render the rays, etc...

[dingsing2021]

how do think about this method?

For the tempory usage, if you want to confirm the current controller type, you can judge whether the position value of the controller is (0, 0, 0). Although this judgment will be late, it is the compromise method provided by the current SDK. Binding 6dof profile can ensure that the controller infos can be obtained in both cases.

[dingsing2021]

feed back from the HVR team: Is there a workaround?We need to add an API to meet the requirements.

[svillar]

feed back from the HVR team: Is there a workaround?We need to add an API to meet the requirements.

Unfortunately the only way we can do it now is to have different builds for 3dof or 6dof because there is no reliable way to detect 3dof or 6dof controllers it at runtime.

[dingsing2021]

maybe there is another way to detect , for 6dof, the usb camera module must be connected to the phone via usb; is there any way to listen or list the usb device , if you can get the special usb camera device(PID/VID)? from android API?

[svillar]

maybe there is another way to detect , for 6dof, the usb camera module must be connected to the phone via usb; is there any way to listen or list the usb device , if you can get the special usb camera device(PID/VID)? from android API?

If anything this should be done in the SDK so that all applications can benefit from that

[dingsing2021]

maybe there is another way to detect , for 6dof, the usb camera module must be connected to the phone via usb; is there any way to listen or list the usb device , if you can get the special usb camera device(PID/VID)? from android API?

If anything this should be done in the SDK so that all applications can benefit from that

yes, but most of other VR APP using untiy or unreal, and they do not need to one apk for 3dof&6dof.

[svillar]

I've just tried .79 and I don't really know what to say here, because I have already mentioned the same in my previous commits. We can create a mixed 3DoF/6DoF APK without any problem by modifying the AndroidManifest.xml file. However, we cannot do anything until #35 is properly fixed because we cannot know at runtime whether we have 3DoF or 6DoF controllers. One example of this is that we don't know which 3D controller model to load to show to the user unless we have a reliable way to know whether we have 3DoF or 6DoF controllers.

[dingsing2021]

right now , vr home launch the wolvic VR by package name : cn.igalia.wolvic.tdof. we hope the package name of wolvic next version , one apk for 3dof&6dof should be same as cn.igalia.wolvic.tdof. how do you think about?

[svillar]

right now , vr home launch the wolvic VR by package name : cn.igalia.wolvic.tdof. we hope the package name of wolvic next version , one apk for 3dof&6dof should be same as cn.igalia.wolvic.tdof. how do you think about?

tdof stands for 3DoF. If we manage to get a single 3dof&6dof package I think we will call it just cn.igalia.wolvic