Inactive controller visualizer after update to 2.7.0

[eusebiu]

Describe the bug

After updating MRTK 2.6.1 to 2.7.0 the controller visualizer gameobject is inactive (e.g. Left_GizmoLeft(Clone)). In 2.6.1 the visualizer is enabled and in sync with the actual controller.

To reproduce

Steps to reproduce the behavior:

1. Update MRTK to 2.7.0
2. Run project with Oculus (Quest 1)
3. Controller visualizer (e.g. Left_GizmoLeft(Clone)) is inactive

Expected behavior

Controller visualizer is active and in sync with controller.

Your setup (please complete the following information)

• Unity 2021.1.12f1
• MRTK 2.7.0

Target platform (please complete the following information)

• Oculus (Quest 1)

[keveleigh]

Unity 2021.1.12f1

Was this previously working with 2.6.1 and Unity 2021? Also, are you able to see anything stand out in the player logs that might point to a cause? I'm wondering if Oculus needs to update their Unity support for 2021 or something.

[eusebiu]

Yes, MRTK 2.6.1 is working fine in Unity 2021. There are no errors/messages in the logs. I am using the latest Oculus Integration (v29, Oculus Utilities v1.61.0, OVRPlugin v1.61.0, SDK v1.61.0)

[keveleigh]

@RogPodge Any ideas on this one?

[RogPodge]

We made changes since 2.6 so that the Oculus controller visualizer defaults to the oculus one when the integration package is present, since we received issues regarding the gizmos being visible even when people didn't want them visible.

Are the Oculus models no longer visible? This PR made it so that the MRTK controller visualization would be disabled when the OVR camera rig gameobject was assigned, since that rig comes with it's own controller models.

If the models in that prefab were disabled, this signals that a gap that we can probably fill with making MRTK more configurable and letting users choose to use the MRTK visualization anyways, even when the oculus one is present.

[eusebiu]

If an user wants to hide the gizmos, it would simple disable the renderer from the gizmo prefab (or else I don't understand what was the initial issue).

In my case I am using the visualizer game object (controller.Visualizer.GameObjectProxy) to parent some object to it using a ParentConstraint and this should work for both hands and controllers (for hands this works fine even in MRTK 2.7.0). If the visualizer is disabled then the "controller" position/rotation is always 0.

So, how can I get a reference to a game object that is linked to the real controller (for both hands and controllers & cross-platform: SteamVR, Oculus, Hololens without going in each platform rig hierarchy)? I used IMixedRealitySourceStateHandler.OnSourceDetected(SourceStateEventData eventData) to get the reference to the MRTK controller and then get the game object of the visualizer, which is now inactive.

So, the GameObjectProxy should normally should point to the Oculus Quest one, no?

[RogPodge]

I think users initially wanted to control the gizmo visibility through the profiles, rather than have to modify their visualizer prefabs directly.

The Visualizer.GameObjectProxy refers to the MRTK visualizer, which is what the PR disabled when the Oculus controllers are available.

Looking at our interface, we do seem to have a gap where we implicitly use the visualizer to communicate the controller's position. @keveleigh can you confirm this? If this is the case, we have a two-fold set of steps we need to take.

1. Exposing position via IMixedRealityController
2. Provide options in the oculus device manager to toggle its own/MRTK's visuals

[eusebiu]

In 2.7.3, OculusControllerVisualization is now internal.

[eusebiu]

For OpenVR/SteamVR the controller.Visualizer.GameObjectProxy is not null... Please add the same oculus object reference to the controller.Visualizer to have backwards compatibility and consistent/cross platform code.

[RogPodge]

Reopening since #10067 never addressed the first point of the two I listed above (sorry about the early issue close!).

[RogPodge]

As of this PR the controller.Visualizer.GameObject Proxy should not be null across all platforms.

In order to get the position of the controller, you should cast either the IMixedRealityController or the IMixedRealityControllerVisualizer to a ControllerPoseSynchronizer. Scripts which are based on that class will respond to OnSourcePoseChanged events, ensuring that it's position matches the input data coming from the input device.

[eusebiu]

Tested with 2.8.0 (from npm) and both visualizers are still null. Also, tried casting to ControllerPoseSynchronizer but it also returned null (on Oculus SDK v39). Just to make sure you understand my use case - I want to use ParentConstraint and for that I need a GameObject/Transform.

Is this going to go back to the functionality from 2.6.3 (-)? If not, they you might as well remove the API. :)

[RogPodge]

The IMixedRealityController should not be null after trying to cast it to a ControllerPoseSyncrhonizer if the controller is tracked and present on the screen. What the PR's have done is parent the Oculus Integration Package's controller visuals to the "Visualizer" object for the controllers.

Are you trying to instantiate the tracking at startup? It may take a frame or 2 for MRTK to parent the visuals in the right places, which might be the cause of the issue.