[Bug] ScheduleAddImageWithValidationJob does not add images to a library

[SimonDarksideJ]

Unity bug report case number https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-23022

Describe the bug A clear and concise description of what the bug is. After much hair pulling, soul searching and cursing, I'm frustrated to state that ScheduleAddImageWithValidationJob simply does not work in the latest Unity 2021 LTS and ARFoundation 4.2.7.

Even using the basic method of

                var mutableLibrary = TrackedImageManager.CreateRuntimeLibrary() as MutableRuntimeReferenceImageLibrary;
                var jobState = mutableLibrary.ScheduleAddImageWithValidationJob(texture, texture.name, 1);
                yield return new WaitUntil(() => jobState.jobHandle.IsCompleted);
                yield return new WaitForEndOfFrame();
                Debug.Log($"{texture.name} has been loaded in to Library, mutable library size [{mutableLibrary.count}]"); // Image added, yet library count is still 0

P.S. Tried using the native method and the result was the same.

The log ALWAYS reports the library count is 0, no matter the image, size or (supported) format.

To Reproduce Steps to reproduce the behavior:

1. Use script above
2. Run a build on iOS or Android
3. Check debug log
4. See error

Expected behavior A clear and concise description of what you expected to happen. When ScheduleAddImageWithValidationJob is used, the tracked image should be processed and added to the XRReferenceImage library.

Actual behavior A clear and concise description of what actually happened. Nothing, nada, lots of time wasted and last trying to get it to work.

Smartphone (please complete the following information):

• Device: Android 11
• Unity version: 2021.3.12
• ARFoundation version: 4.2.7

[SimonDarksideJ]

Tested now in Unity 2022 and ARFoundation 5.0, same results

[SimonDarksideJ]

P.S. Adding the images in manually in the editor seems to work fine. It only fails when adding dynamically.

[tdmowrer]

Adding an image can fail, which is why the object returned from ScheduleAddImageWithValidationJob includes a status property that contains some error states.

I suspect your image is being rejected by ARCore for some reason, and you can confirm whether this is the case by checking the value of status after the job completes. There may also be more information available in logcat (emitted by ARCore with the native tag).

[SimonDarksideJ]

Thanks for the info @tdmowrer , and indeed, after checking the status following the ScheduleAddImageWithValidationJob , it reports "UnknownError". But absolutely no detail on the issue (nothing logged)

I did purposely get it to make an error using none other than the Sample Image included in the ARCore/ARKit packages, which by default fail to load because they are in ASTC6X6 format, which isn't supported

(which also raises an odd query, because those are the default images used in "TestReferenceImageLibraries" INCLUDED with ARCore and ARKit and work when pre-loaded in the editor :S )

Now to make the puzzle more interesting, I updated the DynamicLibrary script included with the ARFoundation samples to be able to submit textures to it for processing, using the update loop instead of the IJob system, and this DID SUCCESSFULLY import the texture and registered it into the MutableLibrary. However, Image Tracking is still unable to detect the image ;(

https://gist.github.com/SimonDarksideJ/71d502cc1f1fa9b50fe5cb6012caf7fe

It would appear that the only way currently to import a texture and have it recognised by the ARFoundation Image Tracking solution, is to have the Image ALREADY loaded into Unity through the editor. If you just try to add it via code, nothing "appears" to work.

[SimonDarksideJ]

The craiziness get weirder, because if I use the above code in a coroutine:

• With 1 image and 1 coroutine run - it fails
• With 2 images and 2 coroutine runs - BOTH work
• WIth 2 images in an array and 1 coroutine run - 1st image fails and 2nd image works

I am agast

[tdmowrer]

Does the image tracking sample included in this repo work? That is, click the "Add Image" button adds images that can be detected? I believe that Unity QA tests (at least) this sample to make sure the dynamic library works prior to each release.

If that works for you, then I suggest you submit a bug so that Unity can try to reproduce the issue with your specific project.

[SimonDarksideJ]

Thanks again @tdmowrer , yes, using the Update Loop method in the DynamicLibrary script in the sample does work, as indicated in

What I did discover however, is that it ONLY works if there is a pre-existing TrackedImageList scriptable asset configured for the ARTrackedImageManager. If the property is null, then image tracking never works (using the ARTrackedImageManager.CreateRuntimeLibrary only).

Logged issues already for most things.

Also logged an issue for the TrackedImage eternal loop issue.

[WyattBUnity]

The public issue is currently in progress. The fix is that it fails because the session state is null when this is executed, the docs will be updated when it is pushed through, but essentially you need to make sure the ARSession.state is SessionTracking or SessionInitializing, or else you may find that the job returns with an error code, via AddReferenceImageJobStatus, which has an enum to show if there was a problem during the job or not.

[ankur-unity]

Closing this issue since the bug is resolved with documentation updates. All three versions: 4.2.8, 5.0.6, and 5.1.0-pre.4 are available now.