MRTK3 and Vuforia bug

[Faith012]

Summarize the bug: Hololens 2 application with MRTK3 and Vuforia Image Target is not working properly. Gameobject from the Image Target is going somewhere very far away.

Observed problem I tried to create an Image Target for an MRTK3 project and when in Play Mode or deployed on Hololens 2 device my Image Target is detected good but the Gameobject which has to be above the image target is displayed normally for a small ammount of time and then is going somewhere very far and when I try in Play Mode to change its position to see it I can find it but it's scaled as a very small gameobject.

Expected result As a result I want to see my child Gameobject from the Image Target parent in the location I set to bet and to be scaled properly.

Reproducible steps Steps to reproduce the behavior:

1. Go to Unity and switch to UWP app and then import all MRTK featuer with Microsoft MixedReality Feature Tool
2. Configure the MRTK project as recommended
3. Add Vuforia Engine and configure as recommended
4. Import Image Target database
5. Add MRTK XR Rig and MRTK Input Simulator
6. Configure main camera and camera offset from MRTK XR Rig as recommended
7. Add a Vuforia Image Target and append a child Gameobject which has to be displayed when image is detected (In my case is a primitive Cube)
8. Run application in Play Mode and try to detect the Image

Affected Vuforia Engine version: v10.22.5

Affected platform:

• Workstation OS: Windows
• Platform Unity
• Platform version 2022.3.22.f1

Affected device:

• Name: Hololens 2

Screenshots / Video Recordings My scene hierararchy:

Additional comments I found from where the problem is coming. In the main camera of the MRTK XR Rig is a component named "Tracked Pose Driver" which has a component called "Update Type" with the default value of Update and Before Render. If the value is changed to just Update then app is working well. My question about this solution is which problems can create this change and what cost is payed for that.

Parameter which has to be changed:

[ptc-epassaro]

Hi @Faith012, I tried following the steps described in the ticket and one of them could be causing your problem. Instead of changing the Update Type on the Tracked Pose Driver component, please try to disable the MRTKInputSimulator. From my observations, it seems that the problem might be caused by a conflict between the Camera poses used in PlayMode by Vuforia and the MRTKInputSimulator. If this is the case, then I would suggest you to always keep the MRTKInputSimulator disabled in the scene every time you want to test a Vuforia feature in PlayMode. For anything else, enable the MRTKInputSimulator, but don't try to detect any Vuforia object.

Regarding the Update Type, since it's a Unity feature I cannot be certain of the internal effects of this option. According to this documentation page, 'Update and Before Render' is the recommended value, as that should provide "the lowest input latency for the device". That said and if you're curious about it, I would recommend you to test the different options on your device and see if the performances are affected in any way. In any case, it shouldn't have any effect on the quality of the tracking in Vuforia Engine.

Please let us know how it goes and if the situation improves.

[Faith012]

HI @ptc-epassaro thanks for a good argumented answer! I tried to disable the MRTK Input Simulator and this strange behaviour disappears. Now since all is working fine I notified a jitter or shaking of my gameobject when the image target is tracked, are there any solutions to get rid of it? I tried to implement some solutions from internet, even tried to apply a median filter to the position of the gameobject but still cannot achieve the desired behaviour that the gameobject to be almost static.

[ptc-epassaro]

Hi @Faith012, great to know that it works! One more tip: if you disable just the Camera simulation in the MRTKInputSimulator you should be able to have Vuforia tracking and input testing at the same time. It's still a bit odd and it takes some getting used to, but it could be something useful for the future, in case you need it.

Regarding the jitter, does this happen in PlayMode or on HoloLens 2? In general, jitter can have multiple causes. The most common is that the scale of the physical target does not match with the image in the database, but there can be more factors. These pages in the Vuforia documentation can be useful:

• Best Practices for Designing and Developing Image-Based Targets
• Image Targets Optimization Techniques
• The Physical Properties of Image-Based Targets
• Enable Device Tracking in Unity

One more thing that can be helpful is to enable pose smoothing on the DefaultObserverEventHandler component:

[Faith012]

Regarding the tip, if I disable Camera simulation then I can simulate the hands and whaterver functionalities I need but only before the target is tracked. After that, simulated hands for example dissapears and my gameobject from the Image Target too, it feels like MRTK Input Simulator wants to overlay inputs over the Vuforia and cannot do that but anyway, the right solution for me remains to deactivate MRTK Input Simulator while I check the functionalities of Vuforia. As for jitter, I saw that I had a typo when I set up the width and after resolving this and added smooth transition, jitter is nearly visible. Thanks a lot for help! I thinked I would never resolve this bug :)

Should I close this issue now?

[ptc-epassaro]

Great to know that the jitter has also improved! Keep in mind that Vuforia cannot use proper Extended Tracking in PlayMode, so the results are always very likely to be different than on device.

Thanks for all the feedback and always feel free to let us know if you encounter any other issue. I'll close the issue now, thanks again!