search

MozillaReality / FirefoxReality

INACTIVE - A fast and secure browser for standalone virtual-reality and augmented-reality headsets.
https://mzl.la/reality
Mozilla Public License 2.0
768 stars 217 forks source link

[HTC] Vive Focus 3 - Controllers and Hands Unavailable #3945

Closed jefmes closed 2 years ago

jefmes commented 3 years ago

Configuration

Firefox Reality version: 12.3 Firefox Reality build ID: fd999304 (AC 51.0.0)

Hardware: HTC Vive Focus 3

Steps to Reproduce

  1. While in headset, launch Firefox Reality
  2. Browse to https://hubs.mozilla.com
  3. Create any room or join any existing room

Current Behavior

The Hub space will open, but the user has no hands, and is unable to move or interact with the environment. Head tracking does appear to work, buy does seem "off" - i.e. overly laggy and while moving it appears as though the user can see the edges of the frame being redrawn.

Expected Behavior

Controllers and/or hands should work! Being able to interact with the environment is very helpful in a virtual world. :)

Possible Solution

Just conjecture on my part, but Firefox Reality may need to be updated to use the latest HTC Wave SDK to properly support the Vive Focus 3. I'm sure you all have more resources about this than I do, but here is the direct link if it's helpful.

https://developer.vive.com/resources/vive-wave/sdk/?site=us

Context

The Focus 3 is now my primary VR headset, and Hubs is essentially unusable in its current state.

Error Logs and Stack Traces

``` ```
rawnsley commented 3 years ago

Are any other WebVR-capable browsers supported on the VF3 for comparison? The problem is unlikely to be in Hubs itself, but somewhere else in the tech stack.

Similarly it might be worth running some simple WebVR demos in Firefox Reality to gauge the level of support.

jefmes commented 3 years ago

No other browser options available at the moment that I'm aware of (but I'm looking into it.) I have run some of the other WebVR and WebXR sites without issue, but I will do a more deliberate pass tomorrow and see if I can narrow down any specific features that aren't functioning correctly. Thanks for quick reply!

jefmes commented 3 years ago

I ran thru the WebXR samples here: https://immersive-web.github.io/webxr-samples/

Overall everything is working, but performance is not great, hovering between 30-40 FPS, which explains some frame rendering problems I'm seeing. The controller model is being recognized as the Vive Focus Plus, which partially works but does not have all the same inputs the Focus 3 controllers have. I will see if I can follow up with HTC and ask about their submitting the proper controller models and profile data to...whomever that is supposed to go to. :)

An interesting side-effect of the testing though, the Vive Cosmos controllers are much closer in button layout minus a menu button on the left controller instead of a Vive button, and in the input tests the Cosmos controller mapped well. Even with the Focus Plus mappings I was able to see the controllers and move around and interact with the environments, so I'm not clear why I'm seeing no inputs at all in Hubs.

Either way, there is clearly more work that needs to be done on the HTC and WebXR side to correctly support the headset. I'll check in with those areas as much I'm able to and see if those resolve the problems I'm seeing in Hubs.

jefmes commented 3 years ago

I may have found the issue or at least part of the issue! Focus 3 profile has not yet been merged.

https://github.com/immersive-web/webxr-input-profiles/pull/199

tolen23 commented 3 years ago

I'm dealing with the exact same problem for a couple of days now.

I also made some tests, but more on the Hubs side, and also created a ticket there (where I also mentioned yours here): https://github.com/mozilla/hubs/issues/4436

From what I found out I fear that this cannot be solved solely on Firefox Reality side. See more details in the ticket above. The good news (at least when you are working with a custom Hubs client in which you can apply code changes): I also found a (dirty?) workaround that is also described there. Maybe this helps you in some way.

jefmes commented 3 years ago

I'm dealing with the exact same problem for a couple of days now.

I also made some tests, but more on the Hubs side, and also created a ticket there (where I also mentioned yours here): mozilla/hubs#4436

From what I found out I fear that this cannot be solved solely on Firefox Reality side. See more details in the ticket above. The good news (at least when you are working with a custom Hubs client in which you can apply code changes): I also found a (dirty?) workaround that is also described there. Maybe this helps you in some way.

Yeah, I realized late last night I had intended this to be in the Hubs repo and not the Firefox Reality repo. Whoops! All in the same family thought, right? I do think there may be something more to this though, because as I posted over in the HTC forums as well, the performance on WebXR apps just does not feel right. I remember even on the Quest 1 most of those experiments ran pretty smoothly. I have not used a Quest 2 but I imagine the XR2 processor should be capable of running these Immersive Web tests at least between 60-90 FPS.

lawwong commented 3 years ago

Suppose the changes #3940 can fix this issue and able to get Focus 3 keys like A/B/X/Y. I haven't able to try the latest version but according to the source code in DeviceDelegateWaveVR, I really can't tell if this changes able to differentiate between Focus Plus and Focus 3 Controller. I'm afraid this will effect the button mapping and rendered model since they have different input profile.

BTW, as far as I know, by calling WVR_GetCurrentControllerModel can identify which device model is used. If it returns string starting with: "WVR_CONTROLLER_FINCH" => Focus 3DoF Controller "WVR_CONTROLLER_ASPEN" => Focus Plus Controller "WVR_CR_Left" => Focus 3 Controller Left "WVR_CR_Right" => Focus 3 Controller Right

jefmes commented 3 years ago

Suppose the changes #3940 can fix this issue and able to get Focus 3 keys like A/B/X/Y. I haven't able to try the latest version but according to the source code in DeviceDelegateWaveVR, I really can't tell if this changes able to differentiate between Focus Plus and Focus 3 Controller. I'm afraid this will effect the button mapping and rendered model since they have different input profile.

BTW, as far as I know, by calling WVR_GetCurrentControllerModel can identify which device model is used. If it returns string starting with: "WVR_CONTROLLER_FINCH" => Focus 3DoF Controller "WVR_CONTROLLER_ASPEN" => Focus Plus Controller "WVR_CR_Left" => Focus 3 Controller Left "WVR_CR_Right" => Focus 3 Controller Right

I think you may be on to something. I've noticed quite a few scenarios where I lose the Focus 3 controllers and either fall back to the Cosmos controllers or the Plus or even Vive Wand controllers. Unrelated to this, even in SteamVR I have been trying to get Focus 3 controllers to show up in a test StereoKit project (using SteamVR's OpenXR runtime) and I have not been able to get hand models to show at all. It does make me think there are problems identifying the controllers overall right now.

jefmes commented 2 years ago

I've tested the Vive Focus 3 today and as of 10/5/2021, hand models are now visible and accurately tracking in Mozilla Hubs. There are still some oddities with navigation and what I can only describe as "involuntary movement" around the space, but the hands are functional. I'm going to go ahead and close this since that was the initial issue. I'd encourage anyone at HTC or Mozilla to get their hands on a Focus 3 and look at this further in Hubs though, because I'm sure there are some other rough edges that could be worked out easily by someone with deeper experience on these systems.