QRCode Tracking Issues [UE5]

[Tim-Potratz]

Hello all, I have been developing with the Hololens for quite a while now and have issues with continues QRCode tracking. Here is what's happening: From time to time, you can see the Debugbox(one that I use to see the tracking position) glitching out to 0, 0, 0 while still being in the "tracking" state. This means your program is going to receive wrong information even with filtering the output. On the other hand, it can work completely fine after the next restart or it can just start happening when the Hololens looses the room.

Here is my tracking code:

Is this a known problem? Did I miss something? If futher informations are needed just ask for it.

Unreal Engine 5.0.3 UXTools 0.12.1 Visual Studio 2022, 17.3.1 Hololens 2, 20348.1522

[luis-valverde-ms]

Hey @Tim-Potratz , this sounds like a platform issue more than a UX Tools one. @fieldsJacksonG , would you be able to help Tim?

[Tim-Potratz]

Are there any updates on this problem? It is some what critical for my app in oder to work.

[luis-valverde-ms]

Sorry for the delay @Tim-Potratz . I've pinged @fieldsJacksonG directly, hopefully we will hear from him soon.

[fieldsJacksonG]

@Tim-Potratz are you using OpenXR or the deprecated WMR plugin?

If you are using WMR, can you switch to OpenXR? You can make this switch by adding this plugin to your game's Plugin directory: https://github.com/microsoft/Microsoft-OpenXR-Unreal/tree/main/MsftOpenXRGame/Plugins/MicrosoftOpenXR

Looking at the WMR QR code, I could see this possibly happening here if the QRCode was not able to be tracked: https://github.com/EpicGames/UnrealEngine/blob/d9d435c9c280b99a6c679b517adedd3f4b02cfd7/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInterop/QRCodeObserver.cpp#L112

Unfortunately, no new WMR changes are going into the engine, and this has been removed in 5.1.

The similar block for OpenXR's QR tracking here should be resilient to this because it more explicitly checks that the QRCode can be located: https://github.com/microsoft/Microsoft-OpenXR-Unreal/blob/55ed0b870125a7151058a3556000603b39bc6f93/MsftOpenXRGame/Plugins/MicrosoftOpenXR/Source/MicrosoftOpenXR/Private/QRTrackingPlugin.cpp#L309

If you are already using OpenXR, I will investigate further!

[Tim-Potratz]

I remember having problems packaging the code with OpenXR plugin, though I will try it out and figure out if it works better.

[Tim-Potratz]

I get this error when trying to use OpenXR:

Should I copy this to the OpenXR issues or did I miss something?

[luis-valverde-ms]

Did you enable the HoloLens plugin? See: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-quickstart#enabling-required-plugins

[Tim-Potratz]

The HoloLens plugin doesn't show up. Maybe because I'm using UE5?

[luis-valverde-ms]

It could be, I'm not that familiar with the latest state there. Did you select HoloLens 2 support in the UE5 installation options? https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-project-setup

[Tim-Potratz]

Yeah I already had done all those steps with WMR, which worked fine. I don't have the Mixed Reality Toolkit Hub installed, because I use the Source plugins to be on the newest state of development. Note that the current OpenXR plugin for UE5 doesn't seem to be officially released. Edit: Okay scrab that last part, I may have downloaded the wrong version, I will try the released one. Edit 2: Nope, still doesn't work.

[camccaf-msft]

Hey Tim, theoretically UXT should work with OpenXR, although it hasn't been tested on UE5 and I don't have a HL2 near me at the moment to test it myself.

I remember having problems packaging the code with OpenXR plugin, though I will try it out and figure out if it works better.

I don't think I've seen packaging issues with OpenXR before, if you can get the error message causing the packaging to fail that would be really useful! If you hit a crash on device when using the MSFT OpenXR plugin, there's a workaround for it here: https://github.com/microsoft/Microsoft-OpenXR-Unreal/issues/84

I get this error when trying to use OpenXR: Should I copy this to the OpenXR issues or did I miss something?

Was that in editor? If so, you can suppress that message by putting the call to Start AR Session behind a branch checking if the result of Is AR Supported is true. The MSFT OpenXR plugin should enable all the required OpenXR engine plugins. But you can double check that the following plugins are enabled:

• OpenXR
• OpenXREyeTracker
• OpenXRHandTracking
• OpenXRMsftHandInteraction

The HoloLens plugin doesn't show up. Maybe because I'm using UE5?

You're correct, the HoloLens platform support was in a plugin in UE4 but has become part of the engine in UE5

Hope that helps!

[Tim-Potratz]

Yes the message is in editor, though it hasn't showed up before and the app crashes on the HoloLens when trying to launch.

[fieldsJacksonG]

The issue Carl linked is most likely why you are crashing, search the blueprints for StartARSession and add a small delay before it is actually called to work around this. If you are using the UXT sample project, StartARSession is called in UXToolsExamples/Common/GameMode/MRGameMode's event graph.

This is fixed in 5.1, but I do not recommend updating right now since HoloLens has a few other 5.1 engine issues that are being resolved before it comes out of preview.

[Tim-Potratz]

Thanks the app starts now and the tracking seems to be okay. Though I can't get the actual tracking state like before.

[Tim-Potratz]

Okay I will move this crashing issue to OpenXR. QRCode tracking works from what I can tell.