How to save Images from MutableRuntimeLibrary to XRReferenceImageLibrary?

[Farid18baku]

I have managed to capture and add images to MutableRuntimeReferenceLibrary. But naturally when I close the app all the images from that library get lost. How do I save the images to the Library so when I open it next time the images are still in library?

Do I have to somehow move images from MutableRuntimeLibrary to XRReferenceLibrary, if yes then how? Since XRReferenceLibrary is not modifiable at runtime.

[Farid18baku]

I found a workaround. Maybe this will help someone Everytime a new Image is added to the Mutable Library I also save its texture separately into a folder in the app. When the user opens the app again I automatically load all the saved textures on Awake one by one using ScheduleAddImageJob. So now user has all the XRReferenceImages he has made during previous sessions. It works.

There is only one weird behavior. When I try to log all the images that are currently in MutableLibrary it only shows the images added during this session, doesn't show images made during previous sessions even if I added them on Awake. But at the same time it clearly finds and reacts to all the images even if doesn't show them in the library.

[woelfle2004]

Hello Farid18baku, it seems you are working on similar things like I do. Please can you share the piece of code how you are able to add images to a MutableRuntimeReferenceLibrary? My problem is that my ReferenceLibrary which is created via Unity inspector is not recognized as a MutableRuntimeReferenceLibrary. Your response is much appreciated, Woelfle2004

[Farid18baku]

Watcht this video, it includes a code for adding images at runtime: https://www.youtube.com/watch?v=af-vY2RnIls&t=268s

[woelfle2004]

Many thanks for your superfast response Farid,I will give it a try, although I thought already tried everything.

[woelfle2004]

I tried this piece of code below but I keep getting the error message "NotSupportedException: No image tracking subsystem found. This usually means image tracking is not supported." when I try to create a ReferenceLibrary by using the function CreateRuntimeLibrary.

Would you please have another hint for me ?

`public class ImageRecognition : MonoBehaviour { private ARTrackedImageManager m_TrackedImageManager; private XRReferenceImageLibrary runtimeImageLibrary;

void Start()
{
    m_TrackedImageManager = gameObject.AddComponent<ARTrackedImageManager>();
    m_TrackedImageManager.referenceLibrary = m_TrackedImageManager.CreateRuntimeLibrary(runtimeImageLibrary);
    m_TrackedImageManager.maxNumberOfMovingImages = 5;
    m_TrackedImageManager.trackedImagePrefab = prefabImage;

    m_TrackedImageManager.enabled = true;
    m_TrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;

    var runtimeReferenceImageLibrary = m_TrackedImageManager.referenceLibrary as MutableRuntimeReferenceImageLibrary;
    Debug.Log("### ImageRecognition " + runtimeReferenceImageLibrary.supportedTextureFormatCount);

    var texture = Resources.Load("IMG") as Texture2D;
    Debug.Log("Texture Name:" + texture.name);

    // if texture exists
    if (texture)
    {
        var addImageJob = runtimeReferenceImageLibrary.ScheduleAddImageJob(texture, texture.name, 0.1f);

        while (!addImageJob.IsCompleted)
        {
            Debug.Log("Image added");
        }

        addImageJob.Complete();
    }

}

}`

[Farid18baku]

1. Have you referenced m_TrackedImageManager and runtimeImageLibrary before the start? As ARTrackedImageManager and XRReferenceImageLibrary.
2. Follow this script , but add this to it: Reference MutableRuntimeReferenceImageLibrary before the start, that worked for me: private MutableRuntimeReferenceImageLibrary mutableRuntimeReferenceImageLibrary; And then set it as Mutable Library: mutableRuntimeReferenceImageLibrary = trackImageManager.referenceLibrary as MutableRuntimeReferenceImageLibrary;

Also I didn't use Start, I used Awake instead. And also run the ScheduleAddImageJob on a button click.

[woelfle2004]

Thank you Farid, that pushed me in the right direction. To complete this discussion you can run through the referenceLibrary by using this code: var enumerator = allg.runtimeReferenceImageLibrary.GetEnumerator(); while (enumerator.MoveNext()) { XRReferenceImage TrackedImage = enumerator.Current; Debug.Log("Tracked Image found: " + TrackedImage.name); }

[tdmowrer]

Thank you Farid, that pushed me in the right direction. To complete this discussion you can run through the referenceLibrary by using this code: var enumerator = allg.runtimeReferenceImageLibrary.GetEnumerator(); while (enumerator.MoveNext()) { XRReferenceImage TrackedImage = enumerator.Current; Debug.Log("Tracked Image found: " + TrackedImage.name); }

That is unnecessarily complex; the GetEnumerator method exists so that you can use a foreach statement:

foreach (var referenceImage in allg.runtimeReferenceImageLibrary)
{
    Debug.Log("Tracked Image found: " + referenceImage.name);
}

Also, note that a reference image (XRReferenceImage) is not the same as a tracked image (ARTrackedImage). A tracked image represents an image that is detected at runtime in the real world, while a reference image is an image to look for. That is, a reference image only describe an image, while a tracked image describes what was detected (position, orientation, etc).

[jiungerich]

Thank you the posters and commentators, this was just what I was looking for. Do any of you know if images can be deleted from the reference image libraries in the same or similar way?

Looking to have a reference image manager feature in my app that lets user add or remove images from the main library. For my app, the AR doesn't have to be currently running as I build image tracking experiences in a separate scene.

[tdmowrer]

Do any of you know if images can be deleted from the reference image libraries in the same or similar way?

No, they can't. We've considered adding this to ARFoundation, but it would only work on ARKit. ARCore does not provide a means to do this, so we can't make it cross-platform.

[jiungerich]

Thank you. I am glad that I asked. I will have to figure a hack.

From want I understand it seems to be possible to just build a whole new library and then replace previous library?
with something like:

var manager = gameObject.AddComponent<ARTrackedImageManager>(); manager.enabled = false; manager.referenceLibrary = myNewLibrary; manager.enabled = true;

I save all the images and image info elsewhere for reference, so it would be possible for me to build a changed library from scratch. This would not be happening very often.

[tdmowrer]

Yep, except you don't need to enable-cycle the ARTrackedImageManager.

[ozankonur]

@tdmowrer I have tried like this way:

    `foreach (LocalResourceModel localResourceModel in localResourceModels) {
            byte[] fileData;

            if (File.Exists (localResourceModel.getResourceLocalUrl ())) {
                fileData = File.ReadAllBytes (localResourceModel.getResourceLocalUrl ());
                texture = new Texture2D (2, 2);
                texture.LoadImage (fileData);

                Debug.Log ("Texture: " + localResourceModel.getResourceName () + " will add to referance library");

                //Add it to library

                Unity.Jobs.JobHandle jobHandle = mutableLibrary.ScheduleAddImageJob (texture, localResourceModel.getResourceName (), 0.3f);
                jobHandle.Complete ();
            }
        }`

But tracked image manager tracks only first image.. Do you think what can be the problem ?

[ozankonur]

Okay... I fixed it with waiting to complete job like this:

       `while (!jobHandle.IsCompleted)
                {
                    Debug.Log("Image added");
                }`

[bakkermark]

Hi posters,

First of all thanks for all the good info on developing an in app image library manager. I conclude based on your info that the only solution to have the images persistent in the image library is to load / add them again after the app is restarted? There is no kind of save method to convert the mutatable image library back to a "normal" image library and that the added images are stored in this converted library?

[tdmowrer]

Hi posters,

First of all thanks for all the good info on developing an in app image library manager. I conclude based on your info that the only solution to have the images persistent in the image library is to load / add them again after the app is restarted? There is no kind of save method to convert the mutatable image library back to a "normal" image library and that the added images are stored in this converted library?

Not currently, but I believe we could add this. I've added this feature request to our backlog.

[bakkermark]

Hi Tim,

That would be really great because then we can develop dynamic AR apps where you can download new images by example from Firebase and sync them with the already downloaded in a previous use of the app. If this can also be done with removing images from the image library (instead of building a image library from scratch) we can manage the image library in the app itself. Thanks in advance!

[mdurand42]

We are tracking this feature request in our backlog. Closing this issue.

[OranBar]

Hi posters, First of all thanks for all the good info on developing an in app image library manager. I conclude based on your info that the only solution to have the images persistent in the image library is to load / add them again after the app is restarted? There is no kind of save method to convert the mutatable image library back to a "normal" image library and that the added images are stored in this converted library?

Not currently, but I believe we could add this. I've added this feature request to our backlog.

Hello. I am very interested in this feature: the ability to save a MutableRuntimeImageLibrary and reload it when the app is restarted.

Was this feature eventually implemented? What's it current status?

Thanks