Detect anchor localization updates multiple times

[Simon-Raes]

We’re working on an AR application in a large venue and are using cloud anchors to perform the user localization. Multiple predefined locations in the building have been scanned and the corresponding locations have been marked in our virtual version of the building. Scanning for and recognizing a cloud anchor allows us to correctly position the digital twin (which encompasses the entire building) at the same location as the cloud anchor.

This works perfectly for the initial localization, but our virtual content is the size of the entire venue and the application will involve the user walking between various points in an undefined order. As this walking causes minor drift, we would like to relocalize and reposition our virtual content ever time azure detects (and relocalizes) a nearby cloud anchor.

It’s at this point where we are running into problems:

• SpatialAnchorManager’s AnchorLocated event is only called once per anchor. If the user walks around the building and returns to the initial anchor, no new event is fired. We would like to know when an anchor’s position is updated so we can correct the drift that was caused by the user walking around the venue.
• Azure automatically links anchors. When the initial anchor is detected, Azure also sends out AnchorLocated events for all other anchors in the area. In combination with the above issue, this makes it impossible for us to relocalize the content when the user reaches the second anchor in the building (as it has already been located at the same time as the initial anchor)
• The application displays elements in mixed reality that cover a large area and can’t be parented to one particular anchor (e.g. we show turn-by-turn navigation elements that may span multiple anchors end-to-end). So we can’t rely on the continuous re-alignment of visible anchor objects that Azure does under the hood (i.e. without raising an event).

What would the best approach be to allow us to keep our digital twin aligned based on encountered cloud anchors?

[msftradford]

Thanks for reaching out @Simon-Raes! Please take a look at our anchor relationships and way-finding in ASA concept guide. When used in a way-finding style app, it can often be advantageous to know where the user is headed next and to occasionally re-query for the pose of the next anchor (by resetting the watcher for each re-query). It is also important to note the differences between the AnchorLocateCriteria that are available. You can always provide the specific ID for the anchor that you want (potentially associated with the next user waypoint) or you can use coarse relocalization to take advantage of additional sensor information (GPS, WiFi, bluetooth) and find anchors based on the user's location in the venue. When a user enters a room, you may want to use coarse relocalization to find the anchors in the room, and then as the user approaches a specific anchor, re-query that anchor by ID. Hope this helps! Please let us know if you have additional question. Thanks!

[Simon-Raes]

@msftradford Thank you for your reply.

When you talk about resetting the watcher or re-querying, are you talking about the SpatialAnchorManager#ResetSessionAsync method? (We're using Unity). Is the idea that we reset the session every time we want to receive new updates to an anchor in our environment?

Is this the only way to receive updates to an anchor's location after the initial recognition and localization of that anchor has happened? Some kind of AchorUpdated callback in addition the AnchorLocated we're already using would be ideal for our use-case.

[Merichbier]

Just dropping in because @Simon-Raes question interest me too.

When I read "re-query" I understand "creating a watcher for the anchor with the ID". Does this really re-position the anchors ? Did I misunderstood the "resetting the watcher" ? I assume that if anchor B was placed thanks to the ASA graph first when detecting anchor A, it might not be at the most accurate position (due to device drift). Would re-querying anchor B with a watcher using Visual features only re-position the anchor more precisely ? Or we need to reset the session ?

Thanks for the help and answer :)

[ephb]

@Simon-Raes For your use case you should look into the AnchorLocateCriteria @msftradford linked to. It would be much better suited to use LocateStrategy VisualInformation.

Relocating anchors that are present in different buildings should work much better this way.

@Merichbier

Would re-querying anchor B with a watcher using Visual features only re-position the anchor more precisely ? Or we need to reset the session ?

Anchors which are already located can not be located again and will be skipped by a watcher.

[Merichbier]

@ephb As I thought, thanks for the answer :)

[msftradford]

Thanks @ephb! I would just add that it is worth looking into and understanding the implications of bypassing the anchor cache. You can use the flag to tell the system to ignore the local cache of anchors and make a call to the service.