When running on iOS 16 the camera feed is unstable (shaky and jittery) when multi-threaded rendering is enabled

[AdamBursill-SV]

Unity bug report case number CASE IN-7240

Describe the bug When running on iOS 16 the camera feed is unstable (shaky and jittery) when multi-threaded rendering is enabled

To Reproduce Steps to reproduce the behavior: Enable multi-threaded rendering and build this samples repo branch 4.1 to an iPhone 12 Pro Max running iOS 16. Open the simple AR scene and start scanning around to generate a few planes, if the camera feed is still stable at this point, lower the application and reopen it. Then the camera feed will go bananas.

Expected behavior I would expect the camera feed to be stable and usable like previous versions of iOS.

Actual behavior The camera feed is jumpy and jittery, also when in this state pressing the return button will crash the app.

Smartphone (please complete the following information):

• Device: iPhone 12 Pro Max, also reproduced with iPhone 11 and iPad Pro M1 however they seemed to require more geometry to get into this strange state
• OS: iOS 16
• Unity version: 2020.3.35f1
• ARFoundation version: 4.1.10

[ankur-unity]

We were able to reproduce the bug internally. You can follow the progress on it at the public issue tracker. https://issuetracker.unity3d.com/issues/ar-camera-feed-jitters-and-shakes-with-mutithreaded-rendering-when-running-on-ios-16

[tinanigro]

Hello, do you have any update to share on this matter?

iOS 16 will probably be released within a month, which means developers will have a short amount of time to update and ship their app with the ARKit plugin update that will fix this issue.

[ankur-unity]

Update: We have a fix for this issue, which will be released in the new patched versions of the packages: 4.1.11, 4.2.5, 5.0.1, as soon as they pass QA validation. https://forum.unity.com/threads/unity-arkit-support-issue-with-ios-16.1334162/

[ankur-unity]

Update: We have released new versions of the packages with a fix for the issue that causes jittering of background texture on iOS 16.

• 5.0.2
• 4.2.6
• 4.1.12

[XubeiX]

I tested the solution in Unity 2021.1.19f on iPhoneX and 12 Pro Max after updating the package to version 4.1.12 the background jittering stopped. But I did note some random app crashes - at the moment I can't say what might be causing them, however Crashlytics noted the fallowing entries.

Crashed: com.Metal.CompletionQueueDispatch 0 CoreFoundation 0x7ba2c CFRelease + 28 1 UnityFramework 0xd63ec4 _ZN12_GLOBALN_121ARKitXRCameraProvider31ScheduleReleaseRetainedTexturesEv_block_invoke + 24 2 Metal 0x20dac MTLDispatchListApply + 44 3 Metal 0xc55c -[_MTLCommandBuffer didCompleteWithStartTime:endTime:error:] + 516 4 IOGPU 0x51b8 -[IOGPUMetalCommandBuffer didCompleteWithStartTime:endTime:error:] + 216 5 Metal 0x20e3c -[_MTLCommandQueue commandBufferDidComplete:startTime:completionTime:error:] + 104 6 IOGPU 0x1192c IOGPUNotificationQueueDispatchAvailableCompletionNotifications + 120 7 IOGPU 0x5f64 __IOGPUNotificationQueueSetDispatchQueue_block_invoke + 60 8 libdispatch.dylib 0x6487c _dispatch_client_callout4 + 16 9 libdispatch.dylib 0x377c0 _dispatch_mach_msg_invoke$VARIANT$armv81 + 372 10 libdispatch.dylib 0x3f72c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 11 libdispatch.dylib 0x3835c _dispatch_mach_invoke$VARIANT$armv81 + 444 12 libdispatch.dylib 0x3f72c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 13 libdispatch.dylib 0x40318 _dispatch_lane_invoke$VARIANT$armv81 + 432 14 libdispatch.dylib 0x3f72c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 15 libdispatch.dylib 0x402e4 _dispatch_lane_invoke$VARIANT$armv81 + 380 16 libdispatch.dylib 0x4a000 _dispatch_workloop_worker_thread + 612 17 libsystem_pthread.dylib 0x1b50 _pthread_wqthread + 284 18 libsystem_pthread.dylib 0x167c start_wqthread + 8

And the second occurring more frequently:

Crashed: com.Metal.CompletionQueueDispatch 0 CoreFoundation 0x80d44 CFRelease + 88 1 UnityFramework 0xd642d4 invocation function for block in (anonymous namespace)::ARKitXRCameraProvider::ScheduleReleaseRetainedTextures() + 383 (ARKitXRCameraProvider.mm:383) 2 Metal 0x22430 MTLDispatchListApply + 52 3 Metal 0xd340 -[_MTLCommandBuffer didCompleteWithStartTime:endTime:error:] + 520 4 IOGPU 0x4b04 -[IOGPUMetalCommandBuffer didCompleteWithStartTime:endTime:error:] + 220 5 Metal 0x224c4 -[_MTLCommandQueue commandBufferDidComplete:startTime:completionTime:error:] + 108 6 IOGPU 0x11c14 IOGPUNotificationQueueDispatchAvailableCompletionNotifications + 128 7 IOGPU 0x594c __IOGPUNotificationQueueSetDispatchQueue_block_invoke + 64 8 libdispatch.dylib 0x409c _dispatch_client_callout4 + 20 9 libdispatch.dylib 0x20830 _dispatch_mach_msg_invoke + 468 10 libdispatch.dylib 0xb56c _dispatch_lane_serial_drain + 376 11 libdispatch.dylib 0x2154c _dispatch_mach_invoke + 448 12 libdispatch.dylib 0xb56c _dispatch_lane_serial_drain + 376 13 libdispatch.dylib 0xc214 _dispatch_lane_invoke + 436 14 libdispatch.dylib 0xb56c _dispatch_lane_serial_drain + 376 15 libdispatch.dylib 0xc1e0 _dispatch_lane_invoke + 384 16 libdispatch.dylib 0x16e10 _dispatch_workloop_worker_thread + 652 17 libsystem_pthread.dylib 0xdf8 _pthread_wqthread + 288 18 libsystem_pthread.dylib 0xb98 start_wqthread + 8

[ankur-unity]

@XubeiX - Thanks for the stack trace. When do you see these crashes, when AR Scene is starting, actively running or exiting?

[AdamBursill-SV]

I have also been testing the fix. I can confirm that the jittery camera feed has been resolved but I am also getting random crashing with the same stack trace. This is while AR is actively running.

Version info: Unity 2021.3.9f1 ARFoundation 4.2.6 Version 14.0 (14A309)

Stack Trace:

thread #59, queue = 'com.Metal.CompletionQueueDispatch', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1a6aacd44) frame #0: 0x00000001a6aacd44 CoreFoundationCFRelease + 88 frame #1: 0x000000010d0f9ad0 UnityFrameworkinvocation function for block in (anonymous namespace)::ARKitXRCameraProvider::ScheduleReleaseRetainedTextures() + 24 frame #2: 0x00000001a1821430 MetalMTLDispatchListApply + 52 frame #3: 0x00000001a180c340 Metal-[_MTLCommandBuffer didCompleteWithStartTime:endTime:error:] + 520 frame #4: 0x000000020d732b04 IOGPU-[IOGPUMetalCommandBuffer didCompleteWithStartTime:endTime:error:] + 220 frame #5: 0x00000001a18214c4 Metal-[_MTLCommandQueue commandBufferDidComplete:startTime:completionTime:error:] + 108 frame #6: 0x000000020d73fc14 IOGPUIOGPUNotificationQueueDispatchAvailableCompletionNotifications + 128 frame #7: 0x000000020d73394c IOGPU__IOGPUNotificationQueueSetDispatchQueue_block_invoke + 64 frame #8: 0x00000001adeb209c libdispatch.dylib_dispatch_client_callout4 + 20 frame #9: 0x00000001adece830 libdispatch.dylib_dispatch_mach_msg_invoke + 468 frame #10: 0x00000001adeb956c libdispatch.dylib_dispatch_lane_serial_drain + 376 frame #11: 0x00000001adecf54c libdispatch.dylib_dispatch_mach_invoke + 448 frame #12: 0x00000001adeb956c libdispatch.dylib_dispatch_lane_serial_drain + 376 frame #13: 0x00000001adeba214 libdispatch.dylib_dispatch_lane_invoke + 436 frame #14: 0x00000001adeb956c libdispatch.dylib_dispatch_lane_serial_drain + 376 frame #15: 0x00000001adeba1e0 libdispatch.dylib_dispatch_lane_invoke + 384 frame #16: 0x00000001adec4e10 libdispatch.dylib_dispatch_workloop_worker_thread + 652 frame #17: 0x00000001f3354df8 libsystem_pthread.dylib_pthread_wqthread + 288

[ankur-unity]

What are your repro steps? Are you using AR Foundation samples or your own project?

I'm unable to reproduce this crash using SimpleAR scene from AR Foundation Samples:

Unity 2021.3.9f1 AR Foundation 4.2.6 Xcode: 14.0 (14A309) iPhone 12 Pro + iOS 16.0 (20A362)

I tried several runs without running into this crash. I also ran the scene for 10 mins straight to check for longer runtime but couldn't get the crash to reproduce.

[XubeiX]

@ankur-unity At this moment I notice the app random crashes only when AR Scene is starting or actively running. I didn't notice this happening when leaving the AR scene or closing the application.

I noticed the problem in my own project, today I will further try to find the cause of the problem.

[AlecCDAS]

@ankur-unity

After installing the updates to ARKit and AR Foundation, the camera feed now runs at an unusable frame rate (about 1 frame every 3 seconds). Will there be any more updates in the near future or have any other users reported this issue?

Unity 2021.3.6f1 ARKit 4.2.6 AR Foundation 4.2.6 Xcode 13.4.1 iPhone 12 Mini iOS 16.0

[ghost]

I'm running 4.1.12 for both ARKit and ARFoundation and that fixed it for me, everything looks normal so far. I'll be testing a lot tomorrow and I'll report back here.

[ankur-unity]

@ankur-unity

After installing the updates to ARKit and AR Foundation, the camera feed now runs at an unusable frame rate (about 1 frame every 3 seconds). Will there be any more updates in the near future or have any other users reported this issue?

@AlecCDAS No, this is not a known issue. 1 frame / 3 sec sounds really low.

1. Is this with the AR Foundation samples or in your own project?
2. Is it only reproducible with ARKit 4.2.6 and not with 4.2.3 for the same project, editor, Xcode, and iPhone 12 Mini?
3. Do you see any errors in the Xcode log window?

[PabloBuitrago]

@ankur-unity

After installing the updates to ARKit and AR Foundation, the camera feed now runs at an unusable frame rate (about 1 frame every 3 seconds). Will there be any more updates in the near future or have any other users reported this issue?

Unity 2021.3.6f1 ARKit 4.2.6 AR Foundation 4.2.6 Xcode 13.4.1 iPhone 12 Mini iOS 16.0

I have this same problem, the camera instability is fixed, but now when I restart the scene with an ARSession the frame rate drops very low (unusable)

Unity 2020.3.34f1 ARKit 4.2.6 and 4.1.12 (I tested both of them) AR Foundation 4.2.6 and 4.1.12 (I tested both of them) Xcode 14.0 iPhone 12 Pro Max iOS 16

(Edit)

I have tried with arfoundation-examples the version for Unity 2022 and 2020, and the problem does not happen. It must be something we don't do like you.

[AlecCDAS]

@ankur-unity

This is in our own project. If I use the version of my project that is on 4.2.3 (same Xcode and device as well) the only issue is the unstable camera feed due to the iOS 16 issue you had already mentioned above.

We have narrowed it down to the XRCameraSubsystem.TryGetLatestFrame() is running significantly slower than in previous versions. If we skip this method, everything runs as intended. Previously this method did not have any performance issues.

https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@2.1/api/UnityEngine.XR.ARSubsystems.XRCameraSubsystem.html#UnityEngine_XR_ARSubsystems_XRCameraSubsystem_TryGetLatestFrame_UnityEngine_XR_ARSubsystems_XRCameraParams_UnityEngine_XR_ARSubsystems_XRCameraFrame__

[AlecCDAS]

@ankur-unity

I have forked the sample repo and added the TryGetLatestFrame() call to the CpuImages scene. The scene was running as intended before adding this method and performance completely tanks after adding it.

https://github.com/AlecCDAS/TryGetLatestFrame_Test

[ankur-unity]

@AlecCDAS - I cannot reproduce the frame rate issue using the AR Foundation Samples with following:

Unity 2021.3.10f1 ARKit 4.2.6 AR Foundation 4.2.6 Xcode 13.4.1 (13F100) iPhone 12 Pro + iOS 16.0

I will try your fork. Thanks for providing that. In the meantime, can you file a bug for it so that there is a public issue tracker to track the progress? https://unity3d.com/unity/qa/bug-reporting

[babinowich]

@ankur-unity I have filed a bug on this exact issue through the Unity interface as suggested but never received a confirmation email. I can't give you any identifying info as I have no email.

[PabloBuitrago]

@ankur-unity

After installing the updates to ARKit and AR Foundation, the camera feed now runs at an unusable frame rate (about 1 frame every 3 seconds). Will there be any more updates in the near future or have any other users reported this issue?

Unity 2021.3.6f1 ARKit 4.2.6 AR Foundation 4.2.6 Xcode 13.4.1 iPhone 12 Mini iOS 16.0

I have this same problem, the camera instability is fixed, but now when I restart the scene with an ARSession the frame rate drops very low (unusable)

Unity 2020.3.34f1

ARKit 4.2.6 and 4.1.12 (I tested both of them)

AR Foundation 4.2.6 and 4.1.12 (I tested both of them)

Xcode 14.0

iPhone 12 Pro Max iOS 16

(Edit)

I have tried with arfoundation-examples the version for Unity 2022 and 2020, and the problem does not happen. It must be something we don't do like you.

After testing for a long time on my project, I have found the cause of the drop in frames. In our application we disable the AR Foundation camera when we are not using it, this causes the frames per second to drop to what is quoted by @AlecCDAS when the AR Foundation camera is enable again.

This has only happened to us with recent AR Foundation versions.

[stetttho]

hi, getting the same crashes as mentioned above running Unity 2021.3.9f1 with ARFoundation 4.2.6 on iOS 16 with iPhone X:

Thread 46 Crashed:
0   libobjc.A.dylib                 0x00000001afac9c88 objc_msgSend + 136
1   UnityFramework                  0x000000010870d0bc invocation function for block in (anonymous namespace)::ARKitXRCameraProvider::ScheduleReleaseRetainedTextures() + 32 (ARKitXRCameraProvider.mm:399)
2   Metal                           0x00000001b12f2dac MTLDispatchListApply + 44 (MTLCommandBuffer.m:59)
3   Metal                           0x00000001b12de55c -[_MTLCommandBuffer didCompleteWithStartTime:endTime:error:] + 516 (MTLCommandBuffer.m:1053)
4   IOGPU                           0x0000000217a691b8 -[IOGPUMetalCommandBuffer didCompleteWithStartTime:endTime:error:] + 216 (IOGPUMetalCommandBuffer.m:157)
5   Metal                           0x00000001b12f2e3c -[_MTLCommandQueue commandBufferDidComplete:startTime:completionTime:error:] + 104 (MTLCommandQueue.m:597)
6   IOGPU                           0x0000000217a7592c IOGPUNotificationQueueDispatchAvailableCompletionNotifications + 120 (IOGPUNotificationQueue.c:331)
7   IOGPU                           0x0000000217a69f64 __IOGPUNotificationQueueSetDispatchQueue_block_invoke + 60 (IOGPUNotificationQueue.c:251)
8   libdispatch.dylib               0x00000001bd1ea87c _dispatch_client_callout4 + 16 (object.m:600)
9   libdispatch.dylib               0x00000001bd1bd7c0 _dispatch_mach_msg_invoke$VARIANT$armv81 + 372 (mach.c:2462)
10  libdispatch.dylib               0x00000001bd1c572c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 (inline_internal.h:0)
11  libdispatch.dylib               0x00000001bd1be35c _dispatch_mach_invoke$VARIANT$armv81 + 444 (mach.c:2784)
12  libdispatch.dylib               0x00000001bd1c572c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 (inline_internal.h:0)
13  libdispatch.dylib               0x00000001bd1c6318 _dispatch_lane_invoke$VARIANT$armv81 + 432 (queue.c:3940)
14  libdispatch.dylib               0x00000001bd1c572c _dispatch_lane_serial_drain$VARIANT$armv81 + 308 (inline_internal.h:0)
15  libdispatch.dylib               0x00000001bd1c62e4 _dispatch_lane_invoke$VARIANT$armv81 + 380 (queue.c:3940)
16  libdispatch.dylib               0x00000001bd1d0000 _dispatch_workloop_worker_thread + 612 (queue.c:6846)
17  libsystem_pthread.dylib         0x00000001fd700b50 _pthread_wqthread + 284 (pthread.c:2618)
18  libsystem_pthread.dylib         0x00000001fd70067c start_wqthread + 8

any help with this would be appreciated.

[PabloBuitrago]

I had many crash problems in the App, until I deactivated Multithreading in Player Settings:

After deactivating it, the crashes of the App were considerably reduced. I runing in iPhone 12 Pro Max

[stetttho]

I had many crash problems in the App, until I deactivated Multithreading in Player Settings:

After deactivating it, the crashes of the App were considerably reduced. I runing in iPhone 12 Pro Max

thanks, but won’t this have a heavy impact on performance? from an official unity tutorial:

You should enable Multithreaded Rendering whenever possible, as it usually benefits performance greatly.

and before updating to iOS 16 I almost never had app crashes.

[XubeiX]

After several days of testing, I can also confirm - disabling multi-threaded rendering significantly reduces the problem. In my case, the problem stopped occurring - Crashlytics has not currently recorded this error once.

Previously in the project I always had multithreaded rendering enabled and there were no problems.

[stetttho]

thanks @PabloBuitrago and @XubeiX for your advice. I now also disabled multi-threaded rendering and so far no more crash happened. Also, I didn’t notice any impact on performance. So going with this until the underlying problem is fixed

[Silloty1]

Hi

Tried to update to the following versions of ARFoundation and ARKit: 4.2.6 4.1.12

They both have low FPS when using it. Only ARFoundation and ARKit version 4.2.3 are providing reaching FPS target.

Config: Unity 2021.3.10 XCode 14 IPhone SE2 with iOS 16

[PabloBuitrago]

Hi

Tried to update to the following versions of ARFoundation and ARKit:

4.2.6

4.1.12

They both have low FPS when using it. Only ARFoundation and ARKit version 4.2.3 are providing reaching FPS target.

Config:

Unity 2021.3.10

XCode 14

IPhone SE2 with iOS 16

In your project do you disable the AR Foundation camera at some point?

[Silloty1]

Hi Tried to update to the following versions of ARFoundation and ARKit: 4.2.6 4.1.12 They both have low FPS when using it. Only ARFoundation and ARKit version 4.2.3 are providing reaching FPS target. Config: Unity 2021.3.10 XCode 14 IPhone SE2 with iOS 16

In your project do you disable the AR Foundation camera at some point?

Hi No I don't, I made sure of that Unfortunately I cannot update to 5.0.2 because of a compatibility with another package using ARFoundation

[derwaldgeist]

I notice the same crashes at ARKitXRCameraProvider::ScheduleReleaseRetainedTextures, especially when using the Face Camera.

[d4n3x]

Has anybody have the issue that with ARfoundation 4.2.6 and Xcode 14.0 to iOS 16.0.2 the project is no longer able to build since there are undefined symbols?

error build: Undefined symbol: _UnityARKit_Camera_GetTextureReleaseCallbackHandle

[matyapav]

Has anybody have the issue that with ARfoundation 4.2.6 and Xcode 14.0 to iOS 16.0.2 the project is no longer able to build since there are undefined symbols?

error build: Undefined symbol: _UnityARKit_Camera_GetTextureReleaseCallbackHandle

@d4n3x Have you updated ARKit XR Plugin to 4.2.6 as well?

[Silloty1]

Hi

Quick update Shaky camera issue is fixed for me with ARFoundation 4.2.3 and iOS 16.0.2 Apple released an update

[tomkrikorian]

Noticed the issue is gone too with ARFoundation 4.2.3 and iOS 16.0.2 but comes back if we update to ARFoundation 4.2.6

[AlecCDAS]

@Silloty1 @tomkrikorian Did you have to change anything else for the fix with iOS 16.0.2? I just tested after updating my device and I see no change at all... 🙁

ARFoundation 4.2.3 and iOS 16.0.2 on iPhone 12 mini.

[ankur-unity]

Hey everyone, I just came back today from a leave and am catching up on the thread. Currently, I’m only able to reproduce the frame rate issue after the jittering patch. We are working to resolve this frame rate issue soon.

Perhaps more details on the issue would help: The original jittering issue happened because with iOS 16, ARKit 6 no longer guarantees that the underlying Metal textures will remain in memory until the rendering of those textures are finished. The fix was to explicitly retain the texture buffers by taking ownership of them. This now happens when a call to XRCameraSubsystem.TryGetLatestFrame() is made. These retained textures will need to be freed once they are no longer needed. We added a step in the command buffer of the rendering pipeline to free the textures after the rendering is complete.

The frame rate issue is happening in the above case because of a separate call to XRCameraSubsystem.TryGetLatestFrame() API as shown in @AlecCDAS’ repro project. A call to this API will need to be followed by a call to release the native metal texture resources. We are looking into providing a way to acquire and release the textures, if necessary. Until we release a patch to fix this issue, we recommend avoiding the use of XRCameraSubsystem.TryGetLatestFrame API separately in your project beyond its usage in ARCameraManager.

[ankur-unity]

Also note that in a recent iOS 16.0.2 patch release, Apple has fixed a similar issue. This fix in iOS could render the patch that Unity implemented to fix jittering, unnecessary.

Camera may vibrate and cause blurry photos when shooting with some third-party apps on iPhone 14 Pro and iPhone 14 Pro Max

Edit (10/07/22): The fix in iOS 16.0.2 does not fix the rendering issue. Unity's patch for fixing the jittering by retaining the frames is still needed.

[babinowich]

@ankur-unity our app requires use of XRCameraSubsystem.TryGetLatestFrame and the iOS 16.0.2 does not fix our issue. Our app is broken until your team fixes this method. Please keep us posted on the fix.

[ankur-unity]

@babinowich - Thanks for your patience while we fix the issue. Until it's fixed, I recommend using AR Foundation 4.1.10, 4.2.3, or 5.0.0-pre.13 without multi-threaded rendering for iOS.

What's your use case for using XRCameraSubsystem.TryGetLatestFrame API?

[AdamBursill-SV]

@ankur-unity we are still hitting the crash issue. The ios 16.0.2 patch has not resolved it. I have tested ARFoundation 4.2.6 with multithreaded rendering disabled and did not see the crash in that scenario. This is not really a solution for our app though as we already push mobile devices to their limits and losing multithreaded rendering is a big problem for us.

I have seen the crash in the samples simple AR scene (with some modifications) but not as repeatable as in our own application. I'm still trying to find an easily repeatable case. You mentioned above that you are now needing to hold onto the metal textures and this line in the stack trace would seem it appears to be related ARKitXRCameraProvider::ScheduleReleaseRetainedTextures().

Have you been able to look into this further? Do you think this could be related to the XRCameraSubsystem.TryGetLatestFrame API issue?

[bH-AGC]

Hi @ankur-unity

Could you tell us when you think this issue would be fixed?

Our solution is currently using Azure Spatial Anchors and they are using XRCameraSubsystem.TryGetLatestFrame in their code.

I just test our solution with the 5.0.0-pre.13 and it works fine but we can't upgrade to Arfoundation 5.0.2 because of this issue!

Hadrien

[mrwellmann]

Our solution is currently using Azure Spatial Anchors and they are using XRCameraSubsystem.TryGetLatestFrame in their code.

@bH-AGC We do have the same issue. Our solution for now is 4.2.3 without multi-threaded rendering as this does not have the issue.

[AdamBursill-SV]

@ankur-unity @andyb-unity Any updates/communication on this issue?

I have managed to get a semi-repeatable case where the application will crash with the above stack trace in the samples repo. Already submitted it: CASE IN-20042. Unfortunately, the issue appears to be a race condition as the app will crash anywhere from 5 minutes to 2 hrs. With our own application, it will crash much sooner. Usually under 10 minutes.

Also found another issue where disabling the AR Camera Background kills position tracking: CASE IN-20043

[ankur-unity]

Thanks for submitting the bug reports, it will be helpful for reproducing these issues for our QA.

We have a solution in review for the frame rate issue. This solution also handles the use case of AR Camera Background being disabled. I am not sure if IN-20043 is a related issue or not. We will have to wait for our QA to triage it.

I'm also hoping that this solution will fix the race condition in reported in IN-20042. I will update here again once the patch for frame rate issue clears testing and is ready for public release.

Note: We have confirmed with Apple that the camera issue which was fixed in iOS 16.0.2 is separate and does not fix the jittering issue. This means that the patch which Unity implemented is still required to fix the rendering.

[ankur-unity]

The public issue tracker for the frame rate issue: https://issuetracker.unity3d.com/issues/ios-fps-in-the-player-is-very-low-when-using-xrcamerasubsystem-dot-trygetlatestframe

The public issue tracker for IN-20043: https://issuetracker.unity3d.com/issues/ios-disabling-ar-camera-background-kills-position-tracking-on-ios-in-arfoundation-4-dot-2-6

[HiRoS-neko]

@bH-AGC , I am now also hitting this issue, what is the current solution you are running with until this gets fixed? and what unity version are you running?

[andyb-unity]

Hi folks,

Patches are rolling out for the TryGetLatestFrame framerate issue and the disabling AR Camera Background kills position tracking issue.

4.1.13: Changelog 4.2.7: Changelog 5.0.3: release in progress. Will update here once that's out as well

Remember to keep version numbers identical for AR Foundation, ARKit, and ARCore packages, and please let us know if you have further issues.

[ankur-unity]

A patch to the current pre-release package is also available:

• 5.1.0-pre.2: https://docs.unity3d.com/Packages/com.unity.xr.arkit@5.1/changelog/CHANGELOG.html#fixed

[ankur-unity]

The patch version 5.0.3 is also available now: https://docs.unity3d.com/Packages/com.unity.xr.arkit@5.0/changelog/CHANGELOG.html#fixed

[ankur-unity]

Closing this issue as it has been resolved in the patches listed above. Feel free to either re-open this issue or create another issue for follow-ups.

[TeorikDeli]

I am using AR Foundation 4.2.7, Unity 2021.3.16f1, iPhone 14 Pro, iOS 16.2. There is no jittering or shaking; but when I enable the multithread rendering, the app crashes randomly. I am suspected environment probes; but disabling it didn't solved the crashes. I couldn't find any usable crash log yet; but xcode shows me these. I will try to disable occlusion manager to see if this is the cause. But is there anyone having a similar random crashes? I am using light estimation, occlusion and environment probes.

EDIT: Yes, the issue seems to related to AROcclusionManager. When I disable occlusion, the app doesn't crashes; but when it is active, the app randomly crashes. Should I open another thread (or send a bug report?), or this could be related to this issue?

[tdmowrer]

Should I open another thread (or send a bug report?)...?

Yes, both :)

[VanIseghemThomas]

Having the same issue as you @TeorikDeli. Can you link the new thread?