ARFoundation 5.0 - Poor tracking stability incl. from samples

[therussmorris]

AR Foundation 5 Unity 2021.3 (LTS) Android 12 device with AR Core support.

Hey.

We have an AR app that we're looking at updating, and at the same time building our AR framework for future projects. The previous app is built on AR Foundation 4.2.4 and in Unity 2021.3, so it's not far off what is the current active releases.

I've been evaluation AR Foundation 5, which I've been doing in Unity 2021.3 (URP 12.1.8). I initially tested in 2022, but had the usual black screen issue so wen't back to 2021 and an older URP version to solve this. as far as I'm aware, ARFoundation 5 should support 2021.3. It gives us some excellent workflow improvements with the simulator, and we also want to make sure we're up to date with latest versions as there are changes to libraries that we'd rather deal with now rather than later.

A blocker currently is the quality of the tracking and the stability of the AR objects. This applies to any AR object, whether it's an AR plane, a tracked image or an anchored object in the world. It's just not performing anywhere near as good as the product we made in AR Foundation 4.2.3.

General tracking is 'okay', things appear where you think they should be, but any particular rotation of the device shows very very noticeable 'drift', particularly when you rotate the device across the horizontal (ie. pointing the device to the sky or the floor). This doesn't happen in our 4.2.3 app, and doesn't happen in other AR apps we've tested on the device (Android AR Elements, Angry Birds etc) - so I don't think there is a problem with this device. I can't show any of our custom anchored objects due to licensing, but attached a video that just shows basic plane tracking. You can see that the planes seem to 'bounce' when rotating the device. It's worth noting that I'm only moving the device every so slightly, the tiniest of amounts, angling them up and down. A matter of 1 or 2 degrees.

This is a video from our test project... https://user-images.githubusercontent.com/683537/212946705-46e42f5d-3271-48a8-ace6-91a0dc7a1367.mp4

This is from the sample project plane detection and object placement... Note the edges of the planes and how much they move up and down

https://user-images.githubusercontent.com/683537/212952081-11d4c7dc-9fb9-4104-b36f-86d9a79ba7aa.mp4

https://user-images.githubusercontent.com/683537/212952282-ac5be997-310a-4c64-aaf0-61fb235670e5.mp4

This is the AR core essential app. There is no movement here...

https://user-images.githubusercontent.com/683537/212952508-c5d00479-3977-46cf-ba20-2156a52ed46f.mp4

I've checked through all the settings and met the requirements, tried multiple locations, and even confirmed the conditions are acceptable - they aren't causing an issue with other apps / our ARFoundation 4.1.2 applications. I've even made sure to try the same markers in the same locations across both the apps and AR Foundation versions on the same phone - it's still solid on 4.2.3 and poor on 5.0

There are various posts about tracking and drift of objects, but some are from old versions or are to do with particular implementation, whereas here I'm seeing it not only in our simple tests, but also in the samples from this git repo.

So my questions right now would be whether what I'm seeing in my project and from the samples expected? Should I expect to see better results in the sample projects? To reiterate, the movements I'm making to the phone are very very minor, as little as I possibly could make. I'd expect a degree of movement with quick, large adjustments, but from very small movements there is a lot of drift - To the point when characters are placed in the AR scene, it looks like they are bouncing or an a trampoline. Any pointers or guidance here would be welcomed.

[angecroft]

We're experiencing the same issue of drifts with ARFoundation 5.0.2 - Unity 2021.3.11f1 on iOS devices (from iPhone 11, iPhone 12 pro, iPhone 14 pro). It is reproducible with AR foundation samples, when you try to see the meshing you can see that it drifts away. This was NOT the case with ARFoundation 4.1.7 (we used it on another project and everything is perfect)

[therussmorris]

@angecroft At least it's not just me experiencing this issue, and to see if happening on another OS is interesting.

Hopefully someone from the team can respond to give an idea whether this is something that is known.

[andyb-unity]

@angecroft this is the first I've heard of any such issue on iOS. Please file a bug? https://unity.com/releases/editor/qa/bug-reporting

@therussmorris We have received multiple reports of ARCore tracking regressions when upgrading from com.unity.xr.arcore 4.2 to 5.0. The most compelling evidence so far comes from this thread where OP provided video of both the 4.2 and 5.0 version of the app: https://forum.unity.com/threads/ar-foundation-v5-0.1327017/#post-8625354

Unity did not change any of our tracking-related code between 4.2 and 5.0. We are working with Google to determine whether the issue might possibly be related to differences between ARCore 1.24 (used in com.unity.xr.arcore 4.2) and ARCore 1.31 (used in com.unity.xr.arcore 5.0).

[angecroft]

Thanks @andyb-unity, I have filed this issue (case 29813).

See the drift on iOS device with ARFoundation 5.0.2 : https://user-images.githubusercontent.com/9963086/214350422-3241d45d-3442-489f-991d-bdb83f21ef77.mov

And from previous version of ARFoundation 4.1.7 https://user-images.githubusercontent.com/9963086/214351688-5e52d024-7118-45bc-8aa2-bb178ba03e59.mp4

[therussmorris]

@andyb-unity Thanks for this info - super useful to know.

[genereddick]

We are seeing the same thing on iOS