search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

OpenVR controllers are not tracked #8835

Closed eusebiu closed 3 years ago

eusebiu commented 3 years ago

Describe the bug

The OpenVR (HTC Vive) controllers are not tracked. The headset is tracked correctly.

To reproduce

Steps to reproduce the behavior:

  1. Install OpenVR XR SDK Plugin https://github.com/ValveSoftware/unity-xr-plugin/releases/tag/installer
  2. Activate OpenVR Loader in XR Plugin Management
  3. Create a profile based on DefaultXRSDKConfigurationProfile - see below
  4. Create an input profile - see below

Expected behavior

Controllers are tracked but they are frozen.

Screenshots

image image

image

Your setup (please complete the following information)

Target platform (please complete the following information)

Additional context

N.A.

keveleigh commented 3 years ago

We haven't yet tested support for Valve's OpenVR XR SDK plug-in. Can you try adding the original OpenVRDeviceManager? It's probable the data is still coming across the legacy input APIs for now, but I had heard that Valve's plug-in doesn't use the XR SDK feature usage APIs for input, which is likely why you're seeing what you're seeing.

eusebiu commented 3 years ago

They also have a bug in the dlls configuration - https://github.com/ValveSoftware/unity-xr-plugin/issues/60 - but with an easy fix.

eusebiu commented 3 years ago

Added but no difference. If I remove the XRSDK Windows Mixed Reality Device Manager, the controllers (gizmo) disappear.

eusebiu commented 3 years ago

https://github.com/ValveSoftware/unity-xr-plugin/issues/16#issuecomment-662708512 Is it me or the Unity XR does not work with this plugin and should use the SteamVR Unity Plugin which installs also the OpenVR Unity Plugin? I tried installing the SteamVR Unity Plugin (2.6.1) but the issue is still present. So, I guess MRTK must change the way it interacts with OpenVR/SteamVR.

eusebiu commented 3 years ago

After some research in SteamVR, it appears that MRTK needs a SteamVR (XR SDK) Device manager & controllers. Using the Windows Mixed Reality XR SDK profile will try to use the Unity XR SDK API which does return any values for pose and input data.

Basically, the inputDevice.TryGetFeatureValue on BaseWindowsMixedRealityXRSDKSource is not workting as it suppose to do. It does work for GenericXRSDKController inputDevice.TryGetFeatureValue(CommonUsages.devicePosition, out CurrentControllerPosition) and rotation but not for BaseWindowsMixedRealityXRSDKSource inputDevice.TryGetFeatureValue(CustomUsages.PointerPosition, out currentPointerPosition) or GenericXRSDKController inputDevice.TryGetFeatureValue(buttonUsage, out bool buttonPressed).

eusebiu commented 3 years ago

A first implementation of this can be seen here: https://github.com/eusebiu/MixedRealityToolkit-Unity/tree/mrtk_development/Assets/MRTK/Providers/SteamVR/XRSDK

stephenprosso commented 3 years ago

The link to your example is broken. We are running into this same issue right now. Can you share your fix to get the controller to track?

eusebiu commented 3 years ago

The link to your example is broken. We are running into this same issue right now. Can you share your fix to get the controller to track?

I've uploaded it here: https://1drv.ms/u/s!Aq1jgNgVNaunk3LGFzgZZItE12Aw?e=9wqLk9

For the sake of folder structure, you should put it in MRTK/Providers. Also, you will need to configure the manager with the SteamVR prefabs (similar to Oculus) - Player can be found here (after you import SteamVR): Assets\SteamVR\InteractionSystem\Core\Prefabs. image

stephenprosso commented 3 years ago

awesome! I followed your instructions and now the controllers track (as VR gloves). I had an error that said I was missing my actions.json file and I was prompted to create one from the example. I did create one and the error went away. The controllers track my hand movements, but I do not have a raycast to select anything. What am I missing to get the ability to raycast and select objects?

I added the OpenVr XR plugin from a tarball file in my package manager. I can see Steam_VR in my assets, but I do not see the SteamVR plugin as imported into my project. Do I still need to import SteamVR or did it come with my OpnVR plugin?

Your SteamVRXRSDKdeviceManager is a lifesaver! thanks for your help!

VR glove image

eusebiu commented 3 years ago

OpenVR plugin (the one from Package manager) is obsolete in the newer versions of Unity (Legacy XR is removed in 2020). So you need to use SteamVR plugin, the one from the Asset Store (which will also be obsolete due to OpenXR). The OpenVR XR plugin (the current one version 1.1.4) is part of the SteamVR Plugin (from asset store) and it will be automatically installed when you import the SteamVR plugin (N.B.: SteamVR plugin is not a plugin in the Unity sense). For raycast you need to configure MRTK to hide the dummy controller - just clone the prefab and disable the Mesh Renderer (that's what I did). In your pictures I see that the ray is visible - see Pointer settings. So, in order to interact with objects, you need to add the MRTK scripts (like Interactable) on the objects you want to interact with.

image image

In my case, the MixedRealityToolkit is configured as (I have support for SteamVR, Oculus and Hololens2 - that's why I have all in the same profile): image image image

eusebiu commented 3 years ago

Also, if you need to override the controls (e.g. to make the touchpad click to behave as the trigger - old VRTK style :) ), you will need to override the SteamVR actions json.

stephenprosso commented 3 years ago

I think my setting match yours, but my raycast still isnt working. Our objects should all be interactible already. Our app works on Oculus and HoloLens2 and now trying to get it deployed to the HTC Vive Pro Eye.

when you say "For raycast you need to configure MRTK to hide the dummy controller - just clone the prefab and disable the Mesh Renderer (that's what I did)." I dont see where to hide the dummy controller. I do also see the raycast on the ground below my avatar but it is not connected to my hand.

In my pointer settings I have "ShellHandRayPointer" selected. I do not have a DefaultControllerPo* as seen in your screen shot. My settings are as follows: 1 2 3 4 5

eusebiu commented 3 years ago

I am referring to these settings. You can clone that prefab and then assign the clone in the profile. In the cloned prefab you can disable the Mesh Renderer component. image

I think the DefaultControllerPointer (the initial name) is the same as ShellHandRayPointer.

I think in your case the controllers are not detected correctly by MRTK - Try running the tool "Controller mappings profiles" from Mixed Reality Toolkit -> Utilities -> Update menu. If that does not work, then you will need to debug the SteamVRXRSDKdeviceManager to see why the controller is not created. Normally, the MRTK controller (the prefab above) should align with and follow the SteamVR one (that comes from SteamVR Player). Also, I forgot to mention that I am using a directive STEAMVR_INTEGRATION that needs to be setup in the Unity Player Project settings (I see that the SteamVR Player needs the HMD Transform): image

stephenprosso commented 3 years ago

To add the STEAMVR_INTEGRATION Do I just add the text as an option or is it like a prefab that i need to drag in or import from somewhere?

Thanks again!

stephenprosso commented 3 years ago

What if we dont want to use the glove hands and we just want to see the controllers?

eusebiu commented 3 years ago

To add the STEAMVR_INTEGRATION Do I just add the text as an option or is it like a prefab that i need to drag in or import from somewhere?

Thanks again!

In Edit -> Project Settings -> Player: image

eusebiu commented 3 years ago

What if we dont want to use the glove hands and we just want to see the controllers?

I think you need to check SteamVR documentation for that - maybe their Player prefab has some settings for changing the hands though I think you will also need to do the rigging, animations, etc. Edit: See in their Player, there is LeftHand/RightHand which have a Hand component which has a Render Model Prefab pointing to the LeftRenderModel Slim which uses vr_glove_left_model_slim. So, you will need to clone the Player prefab and override the hands with the hand you want but you will need to implement the same components.

stephenprosso commented 3 years ago

Is it as easy as just typing in STEAVR_INTEGRATION? Or do i need to import a script from somewhere?

eusebiu commented 3 years ago

Is it as easy as just typing in STEAVR_INTEGRATION? Or do i need to import a script from somewhere?

Just add it in that list - it's simply a preprocessing directive.

eusebiu commented 3 years ago

Will close this as we have a solution by using the custom SteamVR MRTK integration package - https://1drv.ms/u/s!Aq1jgNgVNaunk3LGFzgZZItE12Aw?e=9wqLk9

methot-ilya-3ds commented 2 years ago

From what I've seen this is the closest information there is out here for getting close to making the Vive work with MRTK. For all of the documentation claiming that Vive is supported I've found the opposite to be true. At best the controllers will track but pairing the input system with them seems near impossible.