Crash on ProcessLatestFrame

[tomkrikorian]

Description

Unity crashes when resetting the ARSession (with ARSession.Reset method) when getting the focus of the app back (example : Locking your phone & Unlocking it then resuming the app). The crash happens in the ProcessLatestFrame method of SpatialAnchorManager and when trying to Process the frame.

                if (newFrameToProcess)
                {
                    session.ProcessFrame(xRCameraFrame.nativePtr.GetPlatformPointer());
                    lastFrameProcessedTimeStamp = latestFrameTimeStamp;
                }

Looks like there should be a check to see if the nativePtr is good ?

Development information (please complete the following information)

• OS: Windows\macOS
• Development Platform\Version: Xcode 13.2.1, Unity 2021.1.27f1]
• Azure Spatial Anchors SDK version: 2.11.0

AR Device information (please complete the following information):

• Device: iPhone 11 Mini
• OS: iOS 15.2

Additional context

Here is the backtrace :

Thread 0 name:
Thread 0 Crashed:
0   libsystem_kernel.dylib          0x00000001b7c3ef90 __psynch_cvwait + 8
1   libsystem_pthread.dylib         0x00000001f10a5254 _pthread_cond_wait + 1228 (pthread_cond.c:636)
2   libc++.1.dylib                  0x000000019950bddc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28 (__threading_support:437)
3   AzureSpatialAnchors             0x0000000106aff900 0x106668000 + 4815104
4   AzureSpatialAnchors             0x0000000106b01b8c 0x106668000 + 4823948
5   AzureSpatialAnchors             0x00000001066ccf8c 0x106668000 + 413580
6   AzureSpatialAnchors             0x00000001066c9728 0x106668000 + 399144
7   AzureSpatialAnchors             0x00000001066cb14c 0x106668000 + 405836
8   AzureSpatialAnchors             0x00000001067f2ca8 0x106668000 + 1617064
9   AzureSpatialAnchors             0x00000001067f8118 0x106668000 + 1638680
10  AzureSpatialAnchors             0x000000010678b2e0 0x106668000 + 1192672
11  AzureSpatialAnchors             0x000000010667e144 0x106668000 + 90436
12  UnityFramework                  0x000000010def1c14 NativeLibrary_ssc_cloud_spatial_anchor_session_process_frame_mE763D82346BF7DE8CE4650852A6EDDE8EDFA44F6 + 36 (AzureSpatialAnchors.SDK.iOS.Runtime.cpp:10727)
13  UnityFramework                  0x000000010def1bd0 CloudSpatialAnchorSession_ProcessFrame_m9EC693B8FB46941B378086FF8499DD74E50DF031 + 76 (AzureSpatialAnchors.SDK.iOS.Runtime.cpp:8528)
14  UnityFramework                  0x000000010dedccd0 0x10c5d8000 + 26234064 (AzureSpatialAnchors.SDK.Core.Runtime.cpp:12398)
15  UnityFramework                  0x000000010dedf400 SpatialAnchorManager_ArCameraManager_frameReceived_m3269220229DC0687F9691DB57C5E87D4D1028D08 + 32 (AzureSpatialAnchors.SDK.Core.Runtime.cpp:13742)

[phamqduc]

Hi @tomkrikorian, thank you for sharing with us the issue and your suggestion. Could you please help us to understand how you reset the ARSession when getting the focus of the app back? Are you able to reproduce the issue with the sample code (plus some modification)? Maybe some code snippet and highlighting which operations before and after the app's pause/resume states can help us to reproduce.

[tomkrikorian]

Hi @phamqduc !

So for the full context: Not sure it's relevant but we are using the Unity as a Library feature. So unlike a typical unity application, we don't use the OnApplicationFocus method of a MonoBehavior to know if we got the focus of the app back but we instead call our method to reset our ARSession when we receive the applicationWillEnterForeground event from the native side.

This method basically do this:

                _spatialAnchorManager.DestroySession();
                arSession.Reset();
                await UniTask.WaitUntil(() => ARSession.state == ARSessionState.SessionTracking);
                await StartSession();

I'll try to reproduce using your samples, I cannot reproduce on my side sadly but we received crash reports on TestFlight about this a few times from some of our beta testers.

My current guess is that in some cases SpatialAnchorManager does not properly handle when an ARSession is destroyed and created again. One example is that the ARSessionInitialized member is never set to false.

So if the ARSession is not yet ready at that moment in CreateSessionAsync:

            // Wait for a valid AR Session in case we are starting before AR Foundation is ready.
            if (ARSession.state == ARSessionState.SessionTracking)
            {
                CompleteARFoundationInitialization();
            }

We will never set the session properly again here:

        private void ARSession_stateChanged(ARSessionStateChangedEventArgs obj)
        {
            Debug.Log($"ARSession state changed to {obj.state}");
            if (obj.state == ARSessionState.SessionTracking && !ARSessionInitialized)
            {
                CompleteARFoundationInitialization();
            }
        }

It does not happen in our case because we properly await the ARSession in our code since Day 1 here: await UniTask.WaitUntil(() => ARSession.state == ARSessionState.SessionTracking); But i'm sure if we comment that code, we will get issues.

[phamqduc]

Hi @tomkrikorian, thanks for sharing your insights into how resetting ARSession may have impact on SpatialAnchorManager. ARSessionInitialized poses one issue. I created an internal thread (#37551878) to track it. I tried out replacing CloudManager.ResetSessionAsync in step (case AppState.DemoStepStopSession) with CloudManager.DestroySession(); #if UNITY_ANDROID || UNITY_IOS ARSession arSession = FindObjectOfType<ARSession>(); arSession.Reset(); #endif of the Basic Demo and then re-creating/configuring session as usual. So far, I have not been able to produce a crash yet (unfortunately, testing is only on Unity Android sample - if you suspect different behavior on iOS, please let us know too). Please kindly update us on how you can re-produce the crash on our sample code. Thanks.

[tomkrikorian]

Hi @phamqduc, The issue happens when you lose focus of the app, i'm not sure resetting the ARSession during this step can really reproduce our issue. I'll give you a repro project as soon as i have the time to work on this issue again. We don't have an Android version of our app, i can only confirm that it does happen on iOS so i would suggest to try to reproduce the issue on iOS.

[Varnauld]

Hi @tomkrikorian - apologies for the delay - tracking this under work item 37551878

[nikhilsawlani]

Hi All, I'm facing a similar crash when resetting the AR Session before CreateAnchor

ASA version 2.11.0 Unity - 2020.3.25f1 iOS 15.2, iPhone 13 Pro

The crash happens randomly when creating an Anchor, here are logs

Crashed: UnityGfxDeviceWorker
0  libobjc.A.dylib                0x33f0 objc_retain + 16
1  UnityFramework                 0x9dc884 TexturesMetal::AddCreatedTexture(TextureID, id<MTLTexture>, bool) + 32
2  UnityFramework                 0x94934c GfxDeviceWorker::RunCommand(ThreadedStreamBuffer&) + 13164
3  UnityFramework                 0x9b2974 GfxDeviceWorkerAutoreleasePoolProxy + 68
4  UnityFramework                 0x94ddcc GfxDeviceWorker::RunExt(ThreadedStreamBuffer&) + 92
5  UnityFramework                 0x945ce0 GfxDeviceWorker::RunGfxDeviceWorker(void*) + 12
6  UnityFramework                 0x69ac74 Thread::RunThreadWrapper(void*) + 76
7  libsystem_pthread.dylib        0x19a4 _pthread_start + 148
8  libsystem_pthread.dylib        0xea0 thread_start + 8

[Varnauld]

Hello - could you please try the resolution provided by @msftradford here: https://github.com/Azure/azure-spatial-anchors-samples/issues/284 Please let us know if that resolves your issues, thank you!

[nikhilsawlani]

Hi @Varnauld I fixed the crash by disabling AR Foundation occlusion in Unity for iOS, it seems running AR Foundation occlusion and Azure session at the same time was causing the crash

[Varnauld]

Thank you for the update @nikhilsawlani ! @tomkrikorian are you still having issues?

[Varnauld]

Hello, As we haven't heard back, we will close this issue, but please do reach out if you have any follow up questions or further issues, thank you!