Failed to load XRReferenceImageLibrary : library does not contain any ARCore data when loading from an assetbundle. [Bug]

[pinnakkkk]

Hello Team, I am trying to download an XR image library AssetBundle from the cloud & load it at runtime. The AR Foundation documentation this says it is possible. The code works at runtime in editor, I could open the XR Library & it has all the images & import data. However it doesn't work at runtime on android.

To Reproduce Steps to reproduce the behavior: Code snippet to used to create an assetbundle.

static void BuildAllAssetBundles()
    {
    string assetBundleDirectory = "Assets/AssetBundles/";
    if (!Directory.Exists(Application.streamingAssetsPath))
    {
        Directory.CreateDirectory(assetBundleDirectory);
    }
        BuildPipeline.BuildAssetBundles(assetBundleDirectory, BuildAssetBundleOptions.UncompressedAssetBundle, BuildTarget.Android);
    }

Code to load the assetbundle.

    public IEnumerator DownloadXRlibrary()
    {
        Uri url = new Uri("https:///xrlibrary");

           using (UnityWebRequest request = UnityWebRequestAssetBundle.GetAssetBundle(url))
        {
            yield return request.SendWebRequest();

            if (request.result != UnityWebRequest.Result.Success)
            {
                Debug.LogError(request.error);
                yield break;
            }

              asseBundle = DownloadHandlerAssetBundle.GetContent(request);

            _referenceLibrary = asseBundle.LoadAsset<XRReferenceImageLibrary>("UpdatedReferenceImageLibrary");

            Debug.Log("Loaded Asset: " + _referenceLibrary);

            _arTrackedImageManager.referenceLibrary =  _referenceLibrary;   //breaks here

            _arTrackedImageManager.requestedMaxNumberOfMovingImages = 1;
            _arTrackedImageManager.enabled = true;

            Debug.Log("Loaded Asset's image count: " + _referenceLibrary.count);
            Debug.Log("Image tracker's image count: " + _arTrackedImageManager.referenceLibrary.count);

        }
    }

Expected behavior Downloaded Library should have been loaded & ready for detection.

Actual behavior

2023/02/02 17:14:36.458 23610 23647 Error Unity   at UnityEngine.XR.ARCore.ARCoreImageDatabase..ctor (UnityEngine.XR.ARSubsystems.XRReferenceImageLibrary serializedLibrary) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:36.458 23610 23647 Error Unity   at UnityEngine.XR.ARCore.ARCoreImageTrackingSubsystem+ARCoreProvider.CreateRuntimeLibrary (UnityEngine.XR.ARSubsystems.XRReferenceImageLibrary serializedLibrary) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:36.458 23610 23647 Error Unity   at UnityEngine.XR.ARSubsystems.XRImageTrackingSubsystem.CreateRuntimeLibrary (UnityEngine.XR.ARSubsystems.XRReferenceImageLibrary serializedLibrary) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:36.458 23610 23647 Error Unity   at UnityEngine.XR.ARFoundation.ARTrackedImageManager.set_referenceLibrary (UnityEngine.XR.ARSubsystems.IReferenceImageLibrary value) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:36.458 23610 23647 Error Unity   at PinaksARImageTracker+<ee>d__7.MoveNext () [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:36.458 23610 23647 Error Unity   at U
2023/02/02 17:14:36.480 23610 23759 Error native E0000 00:00:1675338276.480156   23759 motion_tracking_context.cc:1879] NOT_FOUND: Failed to find an image with the requested timestamp.
2023/02/02 17:14:36.480 23610 23759 Error native === Source Location Trace: ===
2023/02/02 17:14:36.480 23610 23759 Error native third_party/redwood/perception/feature_processing/image_buffer.cc:41
2023/02/02 17:14:36.480 23610 23759 Error native third_party/arcore/ar/perception/feature_track_ml_depth_provider.cc:168
2023/02/02 17:14:38.803 23610 23854 Info native I0000 00:00:1675338278.802925   23854 motion_analysis_calculator.cc:650] Analyzed frame 101
2023/02/02 17:14:42.530 23610 23854 Info native I0000 00:00:1675338282.530258   23854 motion_analysis_calculator.cc:650] Analyzed frame 201
2023/02/02 17:14:44.151 23610 23741 Info native I0000 00:00:1675338284.150964   23741 performance_monitor.cc:115] Event: FeatureExtraction is taking too long, it took 111.199ms
2023/02/02 17:14:46.164 23610 23676 Error Unity AssertionException: Assertion failure. Values are equal.
2023/02/02 17:14:46.164 23610 23676 Error Unity Expected: 0 != 0
2023/02/02 17:14:46.164 23610 23676 Error Unity   at UnityEngine.Assertions.Assert.Fail (System.String message, System.String userMessage) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:46.164 23610 23676 Error Unity   at UnityEngine.Assertions.Assert.AreNotEqual[T] (T expected, T actual, System.String message, System.Collections.Generic.IEqualityComparer`1[T] comparer) [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:46.164 23610 23676 Error Unity   at UnityEngine.XR.ARCore.ARCoreImageDatabase.Finalize () [0x00000] in <00000000000000000000000000000000>:0 
2023/02/02 17:14:46.164 23610 23676 Error Unity

Smartphone (please complete the following information):

• Device: Motorola G52
• OS: A13
• Unity version: 2021.3.0f1
• ARFoundation version: 4.2.7

• Attempted Fixes

  1. Tried adding a new AR Image tracking component at runtime.
  2. Tried adding a dummy library with some images while building & then replace it.
  3. Only enabling AR Image tracking component after all it's done attaching all the stuff.

[ondrahracek]

Hi, have you solved this bug or found any workaround? I am facing the same issue on Android. Thanks.

[pinnakkkk]

Hi, have you solved this bug or found any workaround? I am facing the same issue on Android. Thanks.

Not really, it works fine in the editor.

[pinnakkkk]

@tdmowrer could please look into this?

[tdmowrer]

Hi folks -- I no longer work at Unity. I suggest you file an issue if you haven't already.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[scottAVARA]

This is also the issue I am facing. Is there any sort of solution or alternative way of loading reference libraries from assetbundles during runtime?

I have tried the following:

1. Specifying size of target in reference library
2. Checking the 'Keep texture at runtime' check box
3. Ensured all triggers have a high score (75+)

Note: Manually assigning the reference library to the ARTrackedImageManager in the editor and making a build works as expected.

[CarlSustrich]

I'm still finding the same issue, and have been unable to find a workaround for this. Did anyone file an issue and get a helpful response?

[scottAVARA]

I've filed a bug report: IN-52144.

I'll report back once I get a response.

[andyb-unity]

Linking a forum post here to aid the investigator whenever this is assigned: https://forum.unity.com/threads/loading-xrreferenceimagelibrary-through-addressables-runs-in-editor-but-android-builds-crash.1327917/#post-9403268

Thanks, Andy! This looks very similar, but in my case it won't run in editor, either. It gives the error: “InvalidOperationException: Failed to load XRReferenceImageLibrary ‘LIBRARY NAME’: library does not contain any ARCore data.“ is present

IN-55364

[andyb-unity]

Hi all, we finally got this issue assigned to a developer. Sorry that it took so long.

The issue in all cases appears to be this very hidden method called ARBuildProcessor.PreprocessBuild. You must call this method before you build AssetBundles in order for the AssetBundles to contain the necessary image library data.

This is obviously non-intuitive and completely undocumented except for a single comment in the scripting API docs. We are working on a bunch of changes to all versions of AR Foundation to make this easier and better documented, but in the meantime you should be able to test the fix on your end by adding this method call as part of your AssetBundle process.

[andyb-unity]

In AR Foundation 6.0.0-pre.5 we added a new "Build AssetBundles" window that makes building AssetBundles easier and calls ARBuildProcessor.PreprocessBuild for you. More information in our docs.

We have also backported better AssetBundle documentation to older versions, which will be coming out soon.

I'm going to close this issue as fixed, but feel free to re-open it if you have further concerns that are not addressed.

[Bdiebeak]

I encountered the same issue in version 2022.3.20f1. Initially, I thought there might be an issue with the Unity version or the AR Foundation plugin. However, it turned out not to be the case. I tried the same steps in the Unity 6 beta version. In the editor with XR Simulation everything works correctly, but in the build, I have the error XRReferenceImageLibrary library does not contain any ARCore data. Moreover, the error is only visible through Logcat, the build proceeds without any errors or warnings. So, as I understood, there could be two reasons for this problem:

1. You are using Addressables, and before building the bundles themselves, you need to execute ARBuildProcessor.PreprocessBuild - Documentation.
2. Your images are not suitable for use with ReferenceImage because their tracking quality is not high enough.

My issue was with the images. After attempting to build the project using Addressables, according to documentation, I encountered a new error - Failed to get enough keypoints. I realized that the problem might be with the images that were being used. So, I checked them via Arcoreimg Tool and found that one of them resulted in 0% and the other one in 10%. After I replaced the images with new ones that were rated at 100%, and everything started working as expected.