Closed Britvich closed 4 years ago
No, you cannot remove a reference image once it is added to a library (some SDKs do not allow us to do this). What is the use case for doing so?
The documentation you quoted refers to trackables, a different concept than reference images. A reference image is an image to look for in the environment. Once one has been found, the ARTrackedImageManager
generates an ARTrackedImage
, which is a trackable. The ARTrackedImageManager
does not support removing tracked images.
Use case: I want to dynamically look for Snickers bars and Captain Crunch boxes, then after one of them is found, stop looking for one or both of them, and dynamically change the set of looked for images.
Work around: I suppose anytime a reference image is removed (no longer to be looked for), I could just rebuild the entire list of current reference images.
Desired: I wish the API had something like a RemoveImage
call to make it simpler. I also wish I didn't have to use ScheduleAddImageJob
and handle all the job stuff myself. I would prefer just calling AddImage
and let you handle all the gory details behind-the-scenes. ;)
Use case: I want to dynamically look for Snickers bars and Captain Crunch boxes, then after one of them is found, stop looking for one or both of them, and dynamically change the set of looked for images.
Work around: I suppose anytime a reference image is removed (no longer to be looked for), I could just rebuild the entire list of current reference images.
You could also just ignore detected images you no longer care about.
Desired: I wish the API had something like a
RemoveImage
call to make it simpler.
Yes, that would be simpler; however, the underlying SDK has to provide this functionality. ARCore, for instance, has no way to remove reference images from an "augmented image database".
I also wish I didn't have to use
ScheduleAddImageJob
and handle all the job stuff myself. I would prefer just callingAddImage
and let you handle all the gory details behind-the-scenes. ;)
What "job stuff" are you handling? You don't have to do anything other than call ScheduleAddImageJob
unless you really care about knowing when the job completes.
It is called Schedule AddImage Job rather than just "AddImage" to let you know that the work it does is asynchronous, and it provides a JobHandle
in case you want to query for completion or schedule additional work when that job completes. However, you can just call ScheduleAddImageJob
and move on; you don't need to handle anything.
Also note that if you want to add multiple images to the database, the jobs don't have to depend on each other (i.e., you don't have to pass the job handle from one to the next).
Thanks for the response. I'm an old developer and have never seen an API I didn't hate. ;)
I understand some SDKs don't currently support removing reference images. My point is to hide that detail inside your API, where after a RemoveImage
call, the API would either remove it from the reference image set (if the underlying SDK supports that), or just stop reporting detections of that image (if the SDK can't actually remove the image from the set).
Good APIs keep things at the highest level possible, hiding from developers the underlying implementation details.
With ScheduleAddImageJob
, don't I have to wait for it to complete before the ARTrackedImageManager
referenceLibrary
is set and enabled
? I would still prefer an AddImage
call, that produces something like a trackedImagesAdded
event on completion, to make it more symmetrical with the rest of the API and also hide the underlying details of the use of the job system.
Imagine at some point in the future (or it might already be) where the underlying SDK itself asynchronously handles adding images and you (as the API implementer) don't even want to use Unity's job system. Since you've already exposed the use of Unity's job system at the API level, it's hard to no longer use it. The API user (developer) just wants to add/remove images to/from the set and doesn't care how that actually happens.
Thanks again for your ideas and I just wanted to let your know how helpful the AR Foundation has been to my work. I do appreciate all your hard work.
Hello AR explorers :)
Not having an API for removing an image from a mutable image library because one platform doesn't support it would make sense, only if this principle was applied globally in ARFoundation.
But in ARFoundation, it is exactly the opposite. ARFoundation is here to help developers leverage AR features that are common to most platforms while letting them benefit from more advanced capabilities. Here's a list of a few platform-specific capabilities in ARFoundation:
And here I'm only talking about Image Tracking in ARFoundation. There are so many platform-specific features available in ARFoundation, like depth/body occlusion, or object tracking with ARKit. EnvironmentProbes have manual/precise positionning capabilities with ARKit, and limited/automated positionning with ARCore: this difference of capabilities is also well handled in ARFoundation, which is super nice.
These platform-dependant capabilities are specified in the subsystem descriptor by the platform-specific subsystem implementation. This is exactly the way it should be, and it's great to see ARFoundation helping developers get the best of both worlds (common feature-set + pushing the limits on some platforms).
Here are a few suggestions and one potential bug report.
ScheduleRemoveImageJob()
methodAvailable on some platforms.
XRImageTrackingSubsystemDescriptor
adds a new boolean property similar to supportsMutableLibrary
, such as supportsRemovalOfImageInMutableLibrary
(like XREnvironmentProbeSubsystemDescription.supportsRemovalOfManual
).NotSupportedException("This subsystem does not track removing images from the mutable library");
when it's set on unsupported platforms *like XRImageTrackingSubsystem.maxNumberOfMovingImages
which throws throw new NotSupportedException("This subsystem does not track moving images.");
.The ARFoundation team updates the documentation to specify mutable image libraries are only partially mutable, since you cannot properly remove images from that library after adding them.
Our scenario is to be able to track image A, then image A+B+C, then remove image B from tracking. It is currently not possible.
We also tried to track image A, destroy the ARTrackedImageManager
, re-create it, re-configure its mutable image library, and adding the image to the mutable image library once again. What we end up with is only partially working: we properly receive the trackedImagesChanged
callback, but the name of the ARImage and its texture are gone (probably badly cleaned native objects when destroying theARTrackedImageManager
), even though the event itself returns accurate positions and rotations. We are currently investigating thoroughly this issue with a sample project and will submit a ticket if confirmed.
Thanks for you comments, folks. All good suggestions and feedback.
@Britvich
Good APIs keep things at the highest level possible, hiding from developers the underlying implementation details.
I agree that good APIs make simple things simple (which means keeping things high level and hiding implementation details), but they also allow a path for more advanced usage and don't restrict those who want to interact with the implementation details.
With ScheduleAddImageJob, don't I have to wait for it to complete before the ARTrackedImageManager referenceLibrary is set and enabled?
No, you don't have to do anything. It's a fire-and-forget method if you don't care about job completion.
I would still prefer an AddImage call, that produces something like a trackedImagesAdded event on completion, to make it more symmetrical with the rest of the API and also hide the underlying details of the use of the job system.
In Unity, JobHandle
s are the consistent and expected way of interacting with the Unity job system. Not many things in ARFoundation use the job system, so perhaps it feels like a departure from ARFoundation's API surface, but certainly not Unity in general. While an event may be more consistent with the rest of the existing ARFoundation API, do you actually want that? Your original comment on this was
I would prefer just calling AddImage and let you handle all the gory details behind-the-scenes
which, if you ignore the return value, is exactly what ScheduleAddImagJob
gives you.
@ThomasNigro
The XRImageTrackingSubsystemDescriptor adds a new boolean property similar to supportsMutableLibrary, such as supportsRemovalOfImageInMutableLibrary (like XREnvironmentProbeSubsystemDescription.supportsRemovalOfManual).
That is one possibility; we are considering something along these lines.
We also tried to track image A, destroy the ARTrackedImageManager, re-create it, re-configure its mutable image library, and adding the image to the mutable image library once again. What we end up with is only partially working: we properly receive the trackedImagesChanged callback, but the name of the ARImage and its texture are gone (probably badly cleaned native objects when destroying theARTrackedImageManager), even though the event itself returns accurate positions and rotations. We are currently investigating thoroughly this issue with a sample project and will submit a ticket if confirmed.
That sounds like a bug; please submit a bug report and post the case number here if you are able to produce such a sample project.
Hi all! Just to drop my 5 cents in :)
My usecase: I'm developing an app that would allow users to check if the given image is performing well as a trackable image. I.e. the user takes a photo of the image, the app adds it to the reference image library and it immediately tries to detect it; then the user may try different angles, lighting conditions, etc. If the image is good, the user saves the image with metadata, if not, then... I'll ask the user to restart the app before taking another photo :) so that the old photo would not "get in the way" of working with a new one, which may be of the same object. This is an "internal" app and we can "afford" asking users to restart it every time. I'll also try a couple of workarounds that will reinit/recreate TrackableImageManager during runtime. But of course I'd gladly use the opportunity to simply remove the image from the reference library after it was used and saved or discarded.
By the way, thanks a lot - AR Foundation is a real jewel, very cool and easy to work with!
Hi all! Just to drop my 5 cents in :)
My usecase: I'm developing an app that would allow users to check if the given image is performing well as a trackable image. I.e. the user takes a photo of the image, the app adds it to the reference image library and it immediately tries to detect it; then the user may try different angles, lighting conditions, etc. If the image is good, the user saves the image with metadata, if not, then... I'll ask the user to restart the app before taking another photo :) so that the old photo would not "get in the way" of working with a new one, which may be of the same object. This is an "internal" app and we can "afford" asking users to restart it every time. I'll also try a couple of workarounds that will reinit/recreate TrackableImageManager during runtime. But of course I'd gladly use the opportunity to simply remove the image from the reference library after it was used and saved or discarded.
By the way, thanks a lot - AR Foundation is a real jewel, very cool and easy to work with!
You don't need the ability to remove images from a reference image library in order to achieve this. Why not just create a new reference image library for the test image, set that to be the ARTrackedImageManager
's referenceLibrary
, then delete the test reference library if the image is bad?
@tdmowrer thanks a lot! It worked like a charm!
With ScheduleAddImageJob, don't I have to wait for it to complete before the ARTrackedImageManager referenceLibrary is set and enabled?
No, you don't have to do anything. It's a fire-and-forget method if you don't care about job completion.
Actually you have to make sure that you do not destroy the NativeMemory that you had to pass into that method before its actually finished.
Maybe I'm also misunderstand something about the job system here, but ScheduleAddImageJob states clearly that:
// The
So if I want to reduce the memory usage of my application I have to make sure to wait for job completion in order to then clean up the NativeArray I had to pass in.
It would be much easier if the API surface just took any of the image formats that Texture2D can load and handled all the Allocation/Deallocation behind the scenes.
------------- Scratch everything above this line
I just found out that there are indeed Extension methods to the mutable library that provide this API surface. My bad.
@eugeneloza Trying the same thing with ARFoundation 4.0.2 on iOS. But for some reason the system keeps tracking previously added trackables. I'm unsure how to 'delete' a previous reference library. Haven't been able to find a method for it in in the documentation. I'm curious how you got rid of the old trackables in your implementation.
@Hless hi!
I'm curious how you got rid of the old trackables in your implementation.
By replacing the old library with a new one, that contains only one image - that was enough for the usecase. I'll try to chop down everything to a minimal example. Internal variables used:
using Unity.Jobs;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;
...
public ARTrackedImageManager arTrackedImageManager;
RuntimeReferenceImageLibrary newLibrary;
JobHandle jobHandle;
this is actually creating the new library and adding the image to it:
newLibrary = arTrackedImageManager.CreateRuntimeLibrary();
if (newLibrary is MutableRuntimeReferenceImageLibrary mutableLibrary)
{
jobHandle = MutableRuntimeReferenceImageLibraryExtensions.ScheduleAddImageJob(mutableLibrary, imageTexture, imageName, 1);
}
else
{
//log something about impossible to add images and give up
}
finally in Update()
checking every frame if the job is completed:
if (jobHandle.IsCompleted)
{
arTrackedImageManager.referenceLibrary = newLibrary;
arTrackedImageManager.enabled = true;
}
Yes, obviously every image that was in the previous referenceLibrary
is lost. So if you want to keep some of them, you'll need to add them to the new library together with the new images.
@eugeneloza Thank you for the elaborate example! I had almost the same thing implemented, however the old trackables still kept floating around even after reassigning a new library. I'm not sure what the reason is, perhaps its an ARKit issue, or it could be due to the fact I assigned a new library with exactly the same texture as before? (Sometimes I'm loading a different scene, but with the same trackables in the new library).
Anyway, I managed to fix it by calling Reset
on the ARSession before entering a different scene/assigning a new library.
Example code:
var arSession = GetComponent<ARSession>();
arSession.Reset();
Edit:
To specify my problem further: I think it's correct that the system tracks an identical texture with the same trackableId. However, for my implementation I was using _ARTrackedImage.referenceImage.name
as a way to link gameObjects to the trackable. Upon assigning a new library the referenceImage got unset, instead of replaced with the referenceImage in the new library. To me this seems like a bug
Couple quick points to clarify:
@Blackclaws :
Actually you have to make sure that you do not destroy the NativeMemory that you had to pass into that method before its actually finished...So if I want to reduce the memory usage of my application I have to make sure to wait for job completion in order to then clean up the NativeArray I had to pass in.
That's true, but you can schedule a job that deallocates a NativeArray
using the JobHandle
returned by ScheduleAddImageJob
as its dependency. See DeallocateOnJobCompletionAttribute
It would be much easier if the API surface just took any of the image formats that Texture2D can load and handled all the Allocation/Deallocation behind the scenes.
We have exactly that :) See the extension method ScheduleAddImageJob(MutableRuntimeReferenceImageLibrary, Texture2D, String, Nullable<Single>, JobHandle)
@eugeneloza :
if (newLibrary is MutableRuntimeReferenceImageLibrary mutableLibrary) { jobHandle = MutableRuntimeReferenceImageLibraryExtensions.ScheduleAddImageJob(mutableLibrary, imageTexture, imageName, 1); }
This is a very cumbersome way to invoke that method. The point of extension methods is that they, well, extend an existing type. You can just do
jobHandle = mutableLibrary.ScheduleAddImageJob(imageTexture, imageName, 1);
Hello, our program will recognize the picture after entering AR, then exit AR to destroy the AR module. When entering AR for the first time, call the mutableLibrary.ScheduleAddImageJob method to correctly pass in the name of the image, and then in the trackedImagesChanged callback, the name and the GUID have values. After the AR is destroyed, enter again, the name returned by ARTrackedImage is empty, and the GUID is 00000?
Is this what is the reason
@chaoslife I have been struggling with exactly the same issue. See my earlier comment to resolve it: https://github.com/Unity-Technologies/arfoundation-samples/issues/385#issuecomment-650094655
@chaoslife我一直在努力解决同样的问题。请参阅我先前的评论以解决此问题: #385(评论)
Ok,Thanks.I'll try it
Any solution for Bug mention in same Thread ? I am facing same issue
Our scenario is to be able to track image A, then image A+B+C, then remove image B from tracking. It is currently not possible. We also tried to track image A, destroy the ARTrackedImageManager, re-create it, re-configure its mutable image library, and adding the image to the mutable image library once again. What we end up with is only partially working: we properly receive the trackedImagesChanged callback, but the name of the ARImage and its texture are gone (probably badly cleaned native objects when destroying theARTrackedImageManager), even though the event itself returns accurate positions and rotations. We are currently investigating thoroughly this issue with a sample project and will submit a ticket if confirmed.
An alternative to eugeneloza's solution is to to call LoaderUtility.Deinitialize() when the scene is unloaded. This is how Unity handles this in their image tracking sample in AR Foundation Samples.
See the SceneUtility.cs script.
using UnityEngine.SceneManagement;
namespace UnityEngine.XR.ARFoundation.Samples
{
public class SceneUtility : MonoBehaviour
{
void Awake()
{
DontDestroyOnLoad(gameObject);
}
void OnEnable()
{
SceneManager.sceneUnloaded += OnSceneUnloaded;
}
void OnSceneUnloaded(Scene current)
{
if (current == SceneManager.GetActiveScene())
{
LoaderUtility.Deinitialize();
LoaderUtility.Initialize();
}
}
void OnDisable()
{
SceneManager.sceneUnloaded -= OnSceneUnloaded;
}
}
}
I tried LoaderUtility but that won't work in my case !! I am not sure what is the reason. so after research, I got LoaderUtility to use the same class that I use so I went with that.
After dynamically adding a tracked image using
ScheduleAddImageJob
to aMutableRuntimeReferenceImageLibrary
, is there a way to remove that image from the library, so that it is no longer tracked?The docs say:
Is there something like a
RemoveImage
for aMutableRuntimeReferenceImageLibrary
?