FPS dropping below 15 in android platform using WLT

[nRammm]

I'm trying to deploy an MRTK project to android in unity 2020.3.20 version. But I'm also using WLT which seems to be reducing FPS in the profiler below 15. Initially on using WLT v1.5.0 had the same problem and so I installed the latest v1.5.4 ,but still the issue persists.

Steps to reproduce: 1.Create a sample 3D unity project in 2020.3.20 version. 2.Add MRTK 2.7.2 packages 3.Install ARFoundation 4.1.7 (dependency: ARSubsystem -4.1.7) and ARCore XR plugin 4.1.7 4.Add WLT 1.5.4 (tried with 1.5.0 also ,but the same result) and FWE 1.1.1 5.build and run the sample scene configured with MRTK and WLT in android 6.In profiler could see FPS drop(Updateproxy.Update() and ARSession.Update() takes huge time to execute)

Info of mobile used: 1.Name-Samsung Galaxy M31 2.Android version - 11

Screenshot of this issue:

[fast-slow-still]

Hi @nRammm , I'm afraid I don't have access to a Galaxy device at the moment to test this on. Could you tell me what the frame time is without WLT? (Simply delete the WorldLockingManager object from your scene and run again).

Thanks!

[nRammm]

Hi Thanks for your reply , Without WLT its around 30

[fast-slow-still]

Great, so to clarify, you are seeing roughly the same time spent in ARSession.Update() and UpdateProxy.Update()?

Thanks!

[nRammm]

Yeah as far as I observed , without WLT ARSession.Update() takes ~3-4 ms only , whereas with WLT it seems that both ARSession.Update() (~20-30 ms)and UpdateProxy.Update() (~30-40ms) takes huge time and dropping the FPS

[fast-slow-still]

Great! One last question. Is there anything illuminating in the LogCat spew? Would it be possible to capture the LogCat output (filtered to your test app) and attach it here? (I guess that's two last questions.)

[nRammm]

I'm not using any package for logcat because I don't have log messages coming from Android device. So I'm attaching the debug statements in unity console while the android app is running. Will it be okay?

[fast-slow-still]

Great, thanks! I have something I can try. Would you be able to test it? Again, I don't have a Galaxy device, and don't see this issue on my Android test device, so it would be great if you could verify or refute my fix when it's ready.

If this is blocking you, I might suggest switching from XRSDK to ARFoundation anchor subsystem. It's very similar, you just need to add a ARSession and ARSessionOrigin (and optionally an ARAnchorManager) to your MixedRealityPlayspace object, then rerun the Mixed Reality > World Locking Tools > Configure Scene script.

If you try that, please let me know how it goes. Thanks!

[nRammm]

Thank you for your suggestion! I tried out the way you told , but now UpdateProxy.Update() takes ~0.007ms , whereas there are 2 ARSession objects now (the one in MixedRealityPlayspace object and other one created in runtime due to UnityAR camera settings in toolkit profile) , whose update() takes huge time again , inturn FPS is still 15.Also getting some errors in ARAnchor creation.

[fast-slow-still]

Create a GameObject named "AR Session", and add the ARSession component to it. This will allow MRTK to find it and use it instead of creating another one.

Keep the ARSessionOrigin component on the MixedRealityPlayspace object.

[nRammm]

Thank you ! Yeah now there is no duplication in ARSession and also the FPS is 30 now. But what may be the reason for that error debugged "ARAnchorManager failed to create ARAnchor {id}" ?

As a workaround I could see few things,

1.In AnchorManagerARF script in "CreateAnchor(AnchorId id, Transform parent, Pose initialPose)" method , ARAnchorManager.AddAnchor(Pose) is showing deprecated warning , I think this is the reason for the error(Not sure though).

2. So I enabled the macro "WLT_ADD_ANCHOR_COMPONENT" to add the anchor using Addcomponent\() , and now I couldn't see the error in console. Will this be okay to do?

Also two more doubts ,

1. Can this anchor subsystem be switched in runtime from XRSDK to ARF? . If so , will it be a good practice to do so?
2. I could see "NOTE: Do not use the ARF Anchor Subsystem type. If using AR Foundation for your app, still select XRSDK subsystem." in this link https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/DocGen/Documentation/HowTos/XRSDK.html .Why do they say so?

[fast-slow-still]

1.In AnchorManagerARF script in "CreateAnchor(AnchorId id, Transform parent, Pose initialPose)" method , ARAnchorManager.AddAnchor(Pose) is showing deprecated warning , I think this is the reason for the error(Not sure though).

In an unusual move, Unity deprecated the ARAnchorManager.AddAnchor() API and introduced the AddComponent() pattern mid version. I believe AddComponent doesn't work correctly until Unity 2020.3.4f1 or so. Unfortunately, I haven't found a way to conditionally use ARAnchorManager.AddAnchor() before 2020.3.4f1 and the new API after. (There doesn't seem to be a #if UNITY_2020_3_4F1_OR_NEWER.)

The new API will be automatically switched to with Unity 2020.4. But that leaves the question of what to use now. The old API, while deprecated, has seemed to work fine in all versions I've tested. The new API definitely does NOT work for early versions of Unity 2020.3. So I am sticking with the deprecated version through Unity 2020.3.

If you know a way to switch compilation on a minor version, I would be interested to hear about it.

So I enabled the macro "WLT_ADD_ANCHOR_COMPONENT" to add the anchor using Addcomponent() , and now I couldn't see the error in console. Will this be okay to do?

This is fine, as long as you don't need to support older versions of Unity.

Can this anchor subsystem be switched in runtime from XRSDK to ARF? . If so , will it be a good practice to do so?

I am interested in why you would want to do this?

By the way, the next release will have a fix for Android/iOS double ARSession.Update() in XR SDK, so you will be able to get the performance you are seeing now with XR SDK. If you want to try the fix, look at the latest of AnchorManagerXR.cs, especially the following within AnchorManagerXR.UpdateTrackables():

#if !UNITY_ANDROID && !UNITY_IOS
            if (sessionSubsystem != null)
            {
                sessionSubsystem.Update(new XRSessionUpdateParams
                {
                    screenOrientation = Screen.orientation,
                    screenDimensions = new Vector2Int(Screen.width, Screen.height)
                });
            }
#endif // !UNITY_ANDROID && !UNITY_IOS       

I could see "NOTE: Do not use the ARF Anchor Subsystem type. If using AR Foundation for your app, still select XRSDK subsystem." in this link https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/DocGen/Documentation/HowTos/XRSDK.html .Why do they say so?

Apologies, that is stale documentation from when the WLT XRSDK anchor subsystem was more appropriate, even when using ARFoundation. Changes to ARFoundation since then have required a full WLT ARFoundation subsystem. The documentation is just out of date and needs to be fixed.

[nRammm]

I am interested in why you would want to do this?

Because , I want to support UWP as well as Android , so by default the anchor subsystem will be set to XRSDK , only if Android is chosen it should be changed to ARFoundation at runtime.That's why!

Okay Thanks a lot for all your clarifications...with the fix in AnchorManagerXR.cs able to have 30 FPS with ARSession.Update() taking more or less the same time as previous , but UpdateProxy.Update() is much lower now.

[fast-slow-still]

Great, so with the fix in AnchorManagerXR.cs, you are good to go on Android as well as HoloLens, using XRSDK on both?

[nRammm]

Yep hopefully! Thanks!

[fast-slow-still]

By the way, the ARFoundation anchor subsystem and setup works as well on HoloLens as on Android. In fact, using Azure Spatial Anchors requires it.

Let me know if you hit any further trouble. Closing this issue until then.

Thanks for the feedback!