Vuforia ARCamera vs Unity ARFoundation XROrigin Camera Image Target Performance

[eemerse]

Summarize the bug:

Unity's ARFoundation XROrigin Camera (althrough using Vuforia's Behaviour Component) tracks my image much more poorly than Vuforia's ARCamera.

Observed problem Using a simple ImageTarget from a device database. The child prefab that spawns is incredibly jittery and the pose constantly shifts and ticks with Unity's camera. If I use Vuforia's ARCamera, the prefab sits perfectly where I place it.

Expected result A clear and concise description of what you expected to happen.

I expect the ARFoundation camera to track just as well as Vuforia's. If I receive a 5 star augmentable rating on the dev platform, then the ARFoundation camera should be treating it that way - otherwise it really isn't production ready :'{

Other observations Happens on both Android and iOS

Reproducible steps Steps to reproduce the behavior:

1. New URP Scene in Unity
2. Add a Vuforia Camera to scene
3. Add ImageTarget with simple Cube as child

Build and watch tracking

4. Perform the same steps with ARFoundation Camera

Affected Vuforia Engine version: E.g. v10.22.5.

Affected platform:

• Workstation OS: Windows & Mac
• Platform: Unity
• Platform version [e.g. 2022.3.22f1]

Affected device: Samsung S22 Ultra iPhone 15 Pro

Screenshots / Video Recordings

https://drive.google.com/file/d/177pHdtN0ks5jrn3Lq2s9VI45N8zcCQGS/view?usp=sharing - ARFoundation Camera

https://drive.google.com/file/d/16z5tQG1EQ4MpPbhBQHBjyyatnT5nUcsF/view?usp=sharing - Vuforia

Affected database If applicable, share your database / dataset which is affected in the issue you're experiecing.

Device or platform logs Share any logs from the platform or device.

Workaround Is there currently a workaround that you're using?

Additional comments

help :{ Hoping to have a production app for a client completed in 10 weeks. Don't feel comfortable using Vuforia + ARFoundation when the quality is this limited

[ptc-thesbyro]

Hello,

Vuforia Engine is automatically added to the XR Origin's Main Camera at runtime; when a Vuforia Target (e.g., an Image Target) is in the scene.

I suggest to check that this is indeed the case. Could you confirm that the tracking quality difference is also present in Play Mode? During Play Mode, check if the Vuforia Behaviour component is added to the XR Origin's Main Camera GameObject. Check the Console and make sure "Vuforia Started" is logged successfully. Check that the Image Target hasn't been scaled. AR Foundation doesn't do well with content scaling. Verify that your setup is matching that of the Vuforia Engine + AR Foundation guide.

Hope it helps

[eemerse]

Hi @ptc-thesbyro

Play Mode seems to improve slightly, although its very difficult to tell - until you get it on the mobile. Can confirm that the Behaviour Component does get added to the XROrigin camera and can see through console and log files that Vuforia has started and initialised correctly. The ImageTarget scale is at 1,1,1 as defaulted and the child object is set to 0.05, 0.05, 0.05. Can confirm that my setup matches the guide you posted

Here are some results from testing just a basic scene with an ImageTarget and camera.

ARFoundation XROrigin Camera - Android: https://drive.google.com/file/d/18Bsvjt0mbju_zn4JI6OHBTH8iIW-jHaI/view?usp=sharing

• Logcat - https://drive.google.com/file/d/1Tox-B-MSNZRfLzoC-xvMMPm_YRJLFqD8/view?usp=sharing

Vuforia ARCamera - Android: https://drive.google.com/file/d/184Etkb3n7Ho_J_oB-mluTTOvBdbhNg2J/view?usp=sharing

• Logcat - https://drive.google.com/file/d/1RiZLPQieP2mBAXYNiz9bCSPVuddKJ0Qn/view?usp=sharing

ARFoundation XROrigin Camera - iOS: https://drive.google.com/file/d/1inIvSUA9Kh88id3rGeCZzA74WEu26smP/view?usp=sharing

Vuforia ARCamera - iOS: https://drive.google.com/file/d/1y86uWgAkSvWYhGTt1v5LgTCrSzNAxj7u/view?usp=sharing

I did notice there was an error with the Android ARF test:

"2024/05/22 16:41:30.623 16989 17045 Error native E0000 00:00:1716392490.623690 17045 hit_test.cc:426] INTERNAL: No point hit." kept on firing every frame, even though I have successfully tracked the image. I haven't been able to trace this back so far :(

Any help with this would be wicked :D

You can see a noticeable diff with the ARF camera, it kinda messes up the experience a little :(

Thanks

[eemerse]

Update:

Rebuilt and tested again, no longer get the:

"2024/05/22 16:41:30.623 16989 17045 Error native E0000 00:00:1716392490.623690 17045 hit_test.cc:426] INTERNAL: No point hit." error, but the ARF Camera tracking is still poor

[ptc-emaggio]

Is the different behavior reproducible when the object is stationary and one moves the camera around it?

[eemerse]

hi @ptc-emaggio

I'll test now, 2 mins

[eemerse]

@ptc-emaggio @ptc-thesbyro

Vuforia Camera: https://drive.google.com/file/d/18_ved2ZjnCZHrUZJHM22b2evrBLE7r7I/view?usp=sharing

ARF Camera: https://drive.google.com/file/d/18mXKtwx-rs7zhx23lmc9jGcgGHG0-33S/view?usp=sharing

Looks like the same behaviour

[ptc-emaggio]

Indeed the tracking of the Image target is working worse with the AFR camera. I would also expect the target pose status (https://developer.vuforia.com/library/getting-started/pose-status-and-status-info-unity) to flip to EXTENDED_TRACKED. Thanks for the report we will try to reproduce it on our side.

[eemerse]

@ptc-emaggio

Ok mate, nice one. Let me know how it goes or if you need anything to help push it along.

Question: Would your reproductions / fixes come fairly swiftly in terms of timeframe? I have client app due in 10-14 weeks where I was planning on using Vuforia + ARF, and using the usual ARFCamera throughout the app, including the Plane Scanning and Point clouds. If its a lengthy process for your issue / testing approach, I'll just use the Vuforia camera and the Vuforia Ground Planes for a v1 and create a v2 for the client once / if your fixes come into play.

Unless there is a seamless way to switch between the vuforia's camera for image targeting and to the xrorigin camera feed for plane scanning and other arf components?

Thanks

[ptc-emaggio]

While we cannot promise a timeline, if we can reproduce the issue, and a reasonable fix is found, this will be made available in the next release (currently every 6 weeks).

[eemerse]

@ptc-emaggio

Brilliant, thanks for the assist.

[ptc-emaggio]

@eemerse could you please check again that the physical target dimensions shown in the Target Behavior dialog (see attached image) match the real target dimensions?

[eemerse]

sure 1 second

[eemerse]

@ptc-emaggio

physical target dimensions of the image I'm testing with are 85mm x 55mm. I've set the physical width to 0.055 and the height automatically gets set to 0.097779. I've experimented with smaller and larger target dimensions with the same behaviour

[ptc-emaggio]

@eemerse we are able to reproduce the issue and we will be looking for a fix/mitigation. Still be aware that due to some technical limitations of the ARFoundation API, there will always be a gap between ARFoundation and Vuforia AR Camera performance. As mentioned in the notes here with AR foundation you should expect some lag between the camera images and the position of the AR content. This is mostly relevant if your AR experience requires to move the image target in the scene.

[eemerse]

@ptc-emaggio awesome - let me know how you guys get on.

AH I see, that's a shame because the combination of both is such a winning formula :D Luckily, ImageTarget content isn't a massive part of this app, I personally only need the Image to be recognised so that the rest of the app experience can proceed. But I'd love to easily have some content be tracked from the image itself to really bolster the experience.

Is there a situation where I could use Vuforia's camera but with ARFoundations Plane tracking and other tools and features? Or a method to switch between the two cameras smoothly?

Fingers crossed you guys overcome those limitations :)