Control the ARKit's ARSession Configuration

[josefgrunig]

I tried the Basic Image Tracking sample on iOS and noticed that the ARSession is configured with ARWorldTrackingConfiguration. This is not ideal when you want to track only images without tracking the position of the device relative to the world. In this case the ARImageTrackingConfiguration is more appropriate, especially from the performance point of view on old devices like 6G iPads where the Warning [Technique] World tracking performance is being affected by resource constraints [0] popups continuously

Is it possible to force to use the ARImageTrackingConfiguration on the underlying ARSession instance?

I am working on Unity2021.1.22f1 with ARFoundation 4.1.7

Thank you

[tdmowrer]

Yes, the ARImageTrackingConfiguration is supported. AR Foundation will try to pick the "best" configuration based on the features you request and the features supported by each configuration. The image tracking config supports the world-facing camera, rotation only tracking, image detection (obviously), auto focus, and ambient intensity & ambient color lighting features. If you request any other features, it will likely not choose the image tracking config.

So how is the "best" configuration determined? This logic is controlled by the ConfigurationChooser. The default implementation will pick the one that supports the most number of requested features. If there are multiple candidates, we use the configuration descriptor's rank to break ties. Generally, configurations that use less power will have a favorable rank.

You can implement your own ConfigurationChooser if you know that you always want to use a specific configuration descriptor. On ARKit, the ConfigurationDescriptor's identifier is a pointer to the ARConfiguration Objective-C meta-class (the result of [ARImageTrackingConfiguration class] in Objective-C). Your configuration chooser could, for example, iterate through the list of provided configuration descriptors and just always pick the one whose identifier is this pointer. Note that this ARConfiguration was introduced by Apple in iOS 12 and may not be available on all devices. See also this manual entry on the Configuration chooser for more info.

TL;DR there are two options:

1. Configure the appropriate managers (e.g., facing direction in the ARCameraManager, tracking mode in the ARSession, etc) to request only those features that are supported by the image tracking config and hopefully AR Foundation will pick that automatically (where supported).
2. Implement a custom ConfigurationChooser that always picks the ARImageTrackingConfiguration, or some suitable fallback if not supported (maybe just call the default implementation in that case).

[josefgrunig]

Thank you Tim for the clear and exhaustive explanation. I totally missed that part of ARFoundation!

I managed to use the ARImageTrackingConfiguration by setting the Tracking Mode to "Don't Care" or "RotationOnly" on the ARSession. Thank you.

I am still confused about this last setting; what does it mean "Rotation Only". When tracking Images I get 6 DoF ( when images are on screen). Does it mean it tracks device rotations in world space independently of the Image being tracked or not? If Yes, is it possible to not Track World space at all?

I am aiming to use as less resources as possible because I am targeting 6G iPad with A10 chip and they do not perform well with Unity Platform + World Tracking + Image Tracking. Will try to control camera resolution and frame rates to reduce even more the resources consumption

Thank you Tim!

[DavidMohrhardt]

There is a way to do an Orientation only tracking but not while tracking images. Apple's Documentation has 2 configurations that support Image Tracking and both are 6DOF

Both these support image tracking but are 6DOF: ARWorldTrackingConfiguration

ARImageTrackingConfiguration

The following is 3DOF but does not support image tracking: AROrientationTrackingConfiguration

So while it's possible to get only a 3DOF tracking experience, it won't support image tracking. If you wish to only do image tracking then following @tdmowrer's advice is probably your best bet.

[tdmowrer]

I thought it gave you 6dof for the images but 3dof for the device. That's also how face tracking works (6dof faces but 3dof device). That's how we report its runtime capabilities in the configuration descriptor anyway.

[ankur-unity]

@tdmowrer - That's correct. The 3DOF from AROrientationTrackingConfiguration is the rotation of the device only.

[josefgrunig]

I thought it gave you 6dof for the images but 3dof for the device. That's also how face tracking works (6dof faces but 3dof device). That's how we report its runtime capabilities in the configuration descriptor anyway.

@tdmowrer you are right! I made a quick test, with ARImageTrackingConfiguration I get 6 DoF for the Images while tracking. When I loose tracking ( the image goes partially out of screen and tracking state becomes "Limited" ) I still receive rotation updates for the device orientation. In fact the Augmentation is no more consistent if I translate the device, while instead rotating it keeps the augmentation in place.

Thank you for the explanations, it helped a lot to make this clear to me!

Josef