[Bug] Poor detection and tracking quality of image targets added at runtime

[D33pTh0ught]

Description Adding image targets to a MutableRuntimeReferenceImageLibrary works but the detection and tracking is significantly worse compared to image targets added in the Editor. Dynamically added image targets take longer and are harder to be detected and the pose of the tracked object is usally off. This is true for Android as well as iOS.

To Reproduce Make Image Target Library dynamic:

if (_trackedImageManager.descriptor.supportsMutableLibrary)
{
    if (_serializedLibrary != null)
        _mutableImageLibrary = _trackedImageManager.CreateRuntimeLibrary(_serializedLibrary) as MutableRuntimeReferenceImageLibrary;
    else
        _mutableImageLibrary = _trackedImageManager.CreateRuntimeLibrary() as MutableRuntimeReferenceImageLibrary;

    if (_mutableImageLibrary == null)
        Debug.LogWarning("The mutable image target librariy could not be created!");
    else
    {
        _trackedImageManager.referenceLibrary = _mutableImageLibrary;
        _trackedImageManager.enabled = true;
    }
}

Add dynamic Image Targets at runtime:

Unity.Jobs.JobHandle jobHandle = _mutableImageLibrary.ScheduleAddImageWithValidationJob(texture, imageTargetName, widthInMeters).jobHandle;
jobHandle.Complete();

Expected behavior There should not be a difference in tracking quality if the image targets are added at runtime?

Actual behavior The image targets added at runtime are barely usable for detection and not usable for tracking whereas images in statically created libraries perform well in both areas.

Smartphone:

• Device: iPhone XR (14.4.2), Samsung Galaxy A51 (Android 10), Redmi Note 8 (Android 9)
• Unity 2020.1.17f
• ARFoundation 4.1.1

[tdmowrer]

This sample does the same thing and we have not noticed the difference you describe. Does this sample have that problem for you?

One thing to note is that the fidelity of image tracking is very sensitive to the physical image dimensions you provide. Are you sure they are the same between the Editor and runtime contexts?

[D33pTh0ught]

Thanks for the link to the sample project, somehow I missed that. Unfortunately, the issue is the same when using the sample scene. If I add my images there directly in the static library they work fine, however if I add them in the image array of the DynamicLibrary script the detection and pose estimation performs significantly worse (I provided the same and correct image dimensions for both approaches). I could send you the images so you can check them for yourself (they should be treated confidential since they are already in use in a commercial product).

[tdmowrer]

Does the problem only happen with your specific images, or does it also happen with the images in the sample scene?

I could send you the images so you can check them for yourself (they should be treated confidential since they are already in use in a commercial product).

Please log a bug by following the instructions at https://unity3d.com/unity/qa/bug-reporting and post the case ID here.

[D33pTh0ught]

I found out what is causing this issue. The textures to add to the mutable image library at runtime need to have 'Non-Power of 2' set to 'None' if this option is available in the import settings (it is set to 'ToNearest' per default). So it seems my issues with dynamically added image targets was that the images were using the wrong aspect ratio. Setting 'Non-Power of 2' to 'None' is not required for images assigned to a static image target library. Maybe a hint about this special behavior in the docs would be beneficial.