[Bug] on iOS, tracking seems unable to retain the tracking status of an Image Target and causes flickering

[SimonDarksideJ]

Unity bug report case number Please log a bug at https://unity3d.com/unity/qa/bug-reporting and post the case number here.

Describe the bug We've been testing for some time now with ARFoundation Image Tracking on Android/ARCore and with only a few exceptions, everything is tracking fine.

However, we used the same project and built it for iOS and observed the following issues:

• Tracked image detection seems to find the target and then immediately set it to not tracked.
• Tracked Image content (3D models spawned in place of the target) flicker on and off constantly.

To Reproduce Steps to reproduce the behavior:

1. Create a Project configured ARFoundation Image Tracking project
2. Using the following handler for the OnTrackedImages event

        private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
        {
            // Go through all tracked images that have been added 
            // (-> new markers detected) 
            foreach (var newImage in eventArgs.added)
            {
                if (instantiatedPrefabs.TryGetValue(imageReference, out var instantiatedMarker))
                {
                    OnTrackedImageChanged?.Invoke($"{newImage.referenceImage.name} was found and restored");
                    instantiatedMarker.gameObject.SetActive(newImage.trackingState == TrackingState.Tracking);
                    break;
                }

                foreach (var image in trackedImages)
                {
                    if (image.Name.Equals(newImage.referenceImage.name))
                    {
                        // Found a corresponding prefab for the reference image, and it has not been 
                        // instantiated yet > new instance, with the ARTrackedImage as parent 
                        // (so it will automatically get updated when the marker changes in real life) 
                        GameObject newPrefab = Instantiate(prefabObject, newImage.transform);
                        newPrefab.transform.position = new Vector3(newImage.transform.position.x, newImage.transform.position.y + floating_distance, newImage.transform.position.z);
                        // Store a reference to the created prefab 
                        instantiatedPrefabs.Add(newImage.referenceImage.guid, newPrefab);
                    }
                }
            }

            // Disable instantiated prefabs that are no longer being actively tracked
            foreach (var updatedImage in eventArgs.updated)
            {
                if (instantiatedPrefabs.TryGetValue(updatedImage.referenceImage.guid, out var gameObject))
                {
                    bool isTracked = updatedImage.trackingState == TrackingState.Tracking;
                    if (gameObject.activeSelf != isTracked)
                    {
                        gameObject.SetActive(isTracked);
                    }
                }
            }

            // Remove is called if the subsystem has given up looking for the trackable again.
            // (If it's invisible, its tracking state would just go to limited initially).
            // Note: ARCore doesn't seem to remove these at all; if it does, it would delete our child GameObject
            // as well.
            foreach (var removedImage in eventArgs.removed)
            {
                // Destroy the instance in the scene.
                // Note: this code does not delete the ARTrackedImage parent, which was created
                // by AR Foundation, is managed by it and should therefore also be deleted by AR Foundation.
                if (instantiatedPrefabs.TryGetValue(removedImage.referenceImage.guid, out var gameObject))
                {
                    GameObject.Destroy(gameObject);
                    instantiatedPrefabs.Remove(removedImage.referenceImage.guid);
                }
            }
        }

(InstantiatedPrefabs is just a Dictionary of Prefabs)

        private readonly Dictionary<Guid, GameObject> instantiatedPrefabs = new Dictionary<Guid, GameObject>();

3. Build and run the sample on iOS
4. See error/flickering

Expected behavior A clear and concise description of what you expected to happen. Stable tracked prefabs on the Image target.

Actual behavior A clear and concise description of what actually happened.

on iOS, prefab flickers on / off, on Android it is stable.

Smartphone (please complete the following information):

• OS: iOS 13
• Unity version 2021.3.14f1
• ARFoundation version 4.2.7

[SimonDarksideJ]

Issue resolved, essentially on Android

• Tracking is set as tracked when an image is visible
• Tracking is Limited when it is seen but at an angle or "not sure", but still records it as seen
• Tracking is lost when the image cannot be found. on iOS however
• Tracking is set to tracked when it is initially found or as a trackable moves
• Tracking is set to Limited while it is being tracked
• Tracking is lost when the confidence is too low

So the difference in functionality we were seeing was that the foundation sample only checks for "Tracking State = None" and any other state is valid, where our functionality was dealing with all three states.

The resolution is to fall back to the samples implementation and only deal with the None state and all other states are effectively tracking on all platforms, this resolves flicking on iOS where is seems less sure about tracked images.