UI not usable while moving root transform (UI inside a moving vehicule)

[IssueSyncBot]

Original issue opened by:

@scardin

---

Describe the bug

We would like to interact with UI inside a moving vehicule. When animating the root transform of a VR interactive scene the UI becomes barely interactable. We tried to animate the root transformn using the different unity methid Update, LateUpdate, FixedUpdate. While fixed update animation sliding

To reproduce

Steps to reproduce the behavior:

1. Open the MRTKDevTemplate Unity project
2. Select the Canvas example scene
3. Create an empty game gmae object root in the hierarchy
4. Move everyhting as child of this root gameobject
5. Animate the root gameobject using script
6. Interaction in VR is broken (especially slider)

Expected behavior

I expect that the UI can be interacted correctly in VR while moving (as the player is inside a vehicule)

Screenshots

screenshot

video

https://youtu.be/rGhPN-9CaFY

Your setup (please complete the following information)

• Unity Version [e.g. 2022.3.6f1]
• MRTK Version 3 - branch mrtk3 - Commit c86b0e95bdc3bb5b40f60fe9b2571a1b55b0402c

Target platform (please complete the following information)

• OpenXR - Windows Quest link

Animation script

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class RootAnimation : MonoBehaviour
{

    public MODE mode = MODE.NONE;

    public float rotationSpeed = 50.0f;
    public float translationSpeed = 5.0f;

    float time = 0;

    public bool rotate = false;

    public enum MODE
    {
        NONE,
        UPDATE,
        LATEUPDATE,
        FIXEDUPDATE,
    }

    // Update is called once per frame
    void Update()
    {
        if (mode == MODE.UPDATE)
        {
            Animate(Time.deltaTime);
        }
    }

    private void LateUpdate()
    {
        if (mode == MODE.LATEUPDATE)
        {
            Animate(Time.deltaTime);
        }
    }

    private void FixedUpdate()
    {
        if (mode == MODE.FIXEDUPDATE)
        {
            Animate(Time.fixedDeltaTime);
        }
    }

    public void Animate(float deltaT)
    {
        time += deltaT;

        float normTranslation = translationSpeed * deltaT * Mathf.Sin(time);
        transform.Translate(normTranslation, normTranslation, normTranslation);

        if (rotate)
        {
            float normRotation = rotationSpeed * deltaT;
            transform.Rotate(normRotation, normRotation, normRotation);
        }
    }
}

---

ISSUE MIGRATION

Issue migrated from: https://github.com/microsoft/MixedRealityToolkit-Unity/issues/11769

[AMollis]

@shaynie do you think this is fixed this with commit? https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/commit/3e321a9776ebc43390441021db66bf349573c63f

PR from old repo:

• https://github.com/microsoft/MixedRealityToolkit-Unity/pull/11739

Possible dupe:

• https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/issues/87

[AMollis]

Disregard my earlier statement. The @scardin is using a commit after the fix.

[scardin]

@AMollis Thanks for looking at the issue. We have quite a huge project that is built on top of MRTK and this particular issue is really annoying to us in the short term. Any patch or help on how to circumvent the issue is welcome... apart from keeping the root at the same place and moving the world around which is already considered but hell not desired.

[AMollis]

@scardin, can you describe your scenario a bit more. From the description it sounds like you are moving around the XR Origin. What requires you to move the XR Origin?

I suspect the that the reproduction steps don't fully capture your scenario. It sounds like your scenario is in a moving vehicle, and that you're not explicitly changing the XR Origin pose (i.e. not using the above script).

HoloLens 2 has to be put into a "moving platform" mode to function in a moving vehicle. Otherwise, the device's IMU gets confused. From my understanding your app is running on a Quest device, I'm not sure about Quest's support of moving platforms.

[scardin]

In the real world, the user is seated and not really moving. We would like him to be inside a virtual vehicle that moves in the virtual world.

We are moving the XR origin as the user is placed inside the moving vehicle. We want the user to still interact with the virtual UI inside the cockpit.

For our final application, we are using OpenXR with Vive focus 3 and meta quest HMDs, both for native Android builds and Windows applications using streaming.

I could push the scene modification I described to reproduce the bug. You can check in the video I'm just animating a root transform where I parented the whole content of the Canvas example scene from the mrtk3 branch.

[scardin]

@AMollis Any update? This issue is really blocking on our side

[scardin]

It is becoming quite critical for our current production planning. Is it something that can be fixed or circumvented without too much pain?

Any Update? @shaynie @AMollis

[scardin]

Any Update? @AMollis @shaynie

[AMollis]

Any Update? @AMollis @shaynie

@scardin , the XROrigin is not intended to be moved by the application. The origin is placed in the real world by the device, and in the HoloLens case, moved by the device if the user moves too far from start (i.e. Unbounded tracking mode). For your scenario on HoloLens, you'd set the app to use unbounded tracking mode, and put the device into "moving platform" mode. I'm uncertain on the details of Quest devices.

Unfortunately at this time we don't have developers focused on on Quest, so the community will need to support you in this case.

[AMollis]

Tagging @keveleigh and @david-c-kline ... as they have more experience with Quest devices

[scardin]

Thanks @AMollis, This effect is problematic for us in a pure VR environment where the users are inside moving virtual vehicles. I guess our temporary fix is to make the world move around the user, but it becomes rapidly too constraining for our multi-user multi-vehicle use case. Happy to discuss any possible solution or if there are some modification we could eventually contribute to.

[AMollis]

We should test if this reproduces using HoloLens 2 Moving Platform support.