Unable to locate SpatialAnchors

[timgoeij]

Hello,

I am building an application for the HoloLens 1 with Spatial anchors that create and spatial anchors and save the identifier of the Cloud anchors to a REST API and that works fine. But if I get the identifiers from the REST API and try the SDK to locate all the anchors I got the following exception in my logfile:

`ProcessPoseQueryStreamingPoseEvent - Updating SpatialAnchor [38b3f393-cf18-4610-98a6-9bedf92dc7ef]/NeighborhoodAnchor [bffe49d9-a637-4327-8d4c-7603f4dae6a7] with pose found by QueryNeighborhoodDataId [e3fdcb97-d2d3-4620-9c39-ce6d9a8f82ed] UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object) UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]) UnityEngine.Logger:Log(LogType, Object) UnityEngine.Debug:Log(Object) AzureSpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.CloudSpatialAnchorSession:OnLogDebugStaticHandler(UInt64, IntPtr)

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

Creating spatial anchor - 38b3f393-cf18-4610-98a6-9bedf92dc7ef UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object) UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]) UnityEngine.Logger:Log(LogType, Object) UnityEngine.Debug:Log(Object) AzureSpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.CloudSpatialAnchorSession:OnLogDebugStaticHandler(UInt64, IntPtr)

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

Exception thrown at 0x766E30D2 in Hello World.exe: Microsoft C++ exception: Il2CppExceptionWrapper at memory location 0x019CE1D4. OnError - Unexpected error handling location notification UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object) UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]) UnityEngine.Logger:Log(LogType, Object) UnityEngine.Debug:Log(Object) AzureSpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.CloudSpatialAnchorSession:OnLogDebugStaticHandler(UInt64, IntPtr)

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[ShouldFireLocate] Will not locate because no progress has been made since last request UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object) UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]) UnityEngine.Logger:Log(LogType, Object) UnityEngine.Debug:Log(Object) AzureSpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.Unity.SpatialAnchorManager:OnLogDebug(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.OnLogDebugDelegate:Invoke(Object, OnLogDebugEventArgs) Microsoft.Azure.SpatialAnchors.CloudSpatialAnchorSession:OnLogDebugStaticHandler(UInt64, IntPtr)

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)`

I am using the SpatialAnchorManager from the SDK. I use a seperate script that catch all the event from the manager:

`using Microsoft.Azure.SpatialAnchors; using System.Collections; using System.Collections.Generic; using UnityEngine; using HoloLens.Backend.Templates; using System; using System.Linq; using UnityEngine.XR.WSA; using System.Threading.Tasks; using Microsoft.Azure.SpatialAnchors.Unity;

public class AzureSpatialAnchorManager : Singleton { private SpatialAnchorManager spatialAnchorManager; private WorldAnchorManager worldAnchorManager;

private List<WorldAnchorBackend> anchorsFromBackend = new List<WorldAnchorBackend>();

[SerializeField]
private Transform prefab;

protected override void Awake()
{
    spatialAnchorManager = GetComponent<SpatialAnchorManager>();
    worldAnchorManager = GetComponent<WorldAnchorManager>();

    spatialAnchorManager.SessionCreated += OnSessionCreated;
    spatialAnchorManager.SessionStarted += OnSessionStarted;
    spatialAnchorManager.SessionUpdated += OnSessionUpdate;
    spatialAnchorManager.LogDebug += OnLogDebug;
    spatialAnchorManager.Error += OnError;
    spatialAnchorManager.AnchorLocated += OnAnchorLocated;
    spatialAnchorManager.LocateAnchorsCompleted += OnLocateAnchorsCompleted;

    base.Awake();
}

private async void Start()
{
    Debug.Log("Create session");

    await Task.Delay(300);
    await spatialAnchorManager.CreateSessionAsync();
}

private async void OnSessionCreated(object sender, EventArgs e)
{
    if (sender.GetType() == typeof(SpatialAnchorManager))
    {
        Debug.Log("Session Created");
        await spatialAnchorManager.StartSessionAsync();
    }
}

private void OnSessionStarted(object sender, EventArgs e)
{
    Debug.Log("Session Started");
    worldAnchorManager.SubScribeForAllAnchors();
}

private void OnSessionUpdate(object sender, SessionUpdatedEventArgs args)
{
    string feedback = Enum.GetName(typeof(SessionUserFeedback), args.Status.UserFeedback);
    Debug.Log($"feedback: {args.Status.UserFeedback}");
    FindObjectOfType<FeedbackMapper>().SetNewFeedback(feedback);
}

private void OnAnchorLocated(object sender, AnchorLocatedEventArgs args)
{
    if (sender.GetType() == typeof(SpatialAnchorManager))
    {
        switch (args.Status)
        {
            case LocateAnchorStatus.AlreadyTracked: 
                Debug.Log($"Anchor with identifier {args.Identifier} already tracked");
                break;
            case LocateAnchorStatus.Located:
                WorldAnchorBackend backendAnchorForCloudAnchor = anchorsFromBackend.Find(x => x.AnchorId.Equals(Guid.Parse(args.Identifier)));
                CreateObject(backendAnchorForCloudAnchor, args.Anchor);
                break;
            case LocateAnchorStatus.NotLocated:
                Debug.Log($"Anchor with identifier {args.Identifier} not located");
                break;
            case LocateAnchorStatus.NotLocatedAnchorDoesNotExist:
                Debug.Log($"Anchor with identifier {args.Identifier} not located and does not exists");
                break;
        }
    }
}

private void OnLocateAnchorsCompleted(object sender, LocateAnchorsCompletedEventArgs args)
{
    if (sender.GetType() == typeof(SpatialAnchorManager))
    {
        Debug.Log("Locating anchors completed");
    }
}

private void OnError(object sender, SessionErrorEventArgs args)
{
    Debug.LogError($"Code: {args.ErrorCode}, Message: {args.ErrorMessage}");
}

private void OnLogDebug(object sender, OnLogDebugEventArgs args)
{
    Debug.Log(args.Message);
}

public void GetAnchorsWithIds(List<WorldAnchorBackend> backendAnchors)
{
    Debug.Log("Get Ids from server");

    if (!backendAnchors.Any())
    {
        Debug.Log("There are currently no anchors in the database");
        return;
    }

    anchorsFromBackend = backendAnchors;

    if(backendAnchors.Count <= 35)
    {
        AnchorLocateCriteria anchorLocateCriteria = new AnchorLocateCriteria();
        anchorLocateCriteria.Identifiers = backendAnchors.Select(x => x.AnchorId.ToString()).ToArray();
        Debug.Log($"Create Watcher for {anchorLocateCriteria.Identifiers.Length} anchors");
        spatialAnchorManager.Session.CreateWatcher(anchorLocateCriteria);
    }
    else
    {
        int startIndex = 0;

        while(startIndex < backendAnchors.Count)
        {
            AnchorLocateCriteria anchorLocateCriteria = new AnchorLocateCriteria();

            anchorLocateCriteria.Identifiers = backendAnchors
                .Select(x => x.AnchorId.ToString())
                .Skip(startIndex)
                .Take(35)
                .ToArray();

            Debug.Log($"Create Watcher for {anchorLocateCriteria.Identifiers.Length} anchors");
            spatialAnchorManager.Session.CreateWatcher(anchorLocateCriteria);
            startIndex += 35;
        }
    }   

    Debug.Log($"Is session locating anchors: {spatialAnchorManager.IsLocating}");
}

public async void SaveAnchor(WorldAnchorBackend backendAnchor)
{
    Debug.Log("Creating a new spatial anchor");
    BaseElement newElement = CreateObject(backendAnchor);

    await spatialAnchorManager.CreateAnchorAsync(newElement.CloudSpatialAnchor);
    Debug.Log($"Identifier of spatial anchor: {newElement.CloudSpatialAnchor.Identifier}");

    if (newElement.CloudSpatialAnchor != null)
    {
        newElement.UpdateWorldAnchor(newElement.WorldAnchor, newElement.CloudSpatialAnchor);
        newElement.BackendWorldAnchorPoint.AnchorId = newElement.CloudSpatialAnchor.Identifier;
        FindObjectOfType<WorldAnchorManager>().SaveWorldAnchor(newElement.BackendWorldAnchorPoint, newElement.WorldAnchor);
    }
}

private BaseElement CreateObject(WorldAnchorBackend worldAnchorBackend)
{
    CloudSpatialAnchor cloudSpatialAnchor = new CloudSpatialAnchor();
    return CreateObject(worldAnchorBackend, cloudSpatialAnchor, true);
}

private BaseElement CreateObject(WorldAnchorBackend worldAnchorBackend, CloudSpatialAnchor cloudSpatialAnchor, bool hasNotAnAnchor = false)
{
    Transform newElement = Instantiate(prefab) as Transform;

    BaseElement baseElement = newElement.GetComponent<BaseElement>();
    baseElement.SetHoloLensWorldAnchorPoint(worldAnchorBackend);

    WorldAnchor worldAnchor = baseElement.AddWorldAnchor();

    if(hasNotAnAnchor)
    {
        Debug.Log("Create without a cloud anchor");
        cloudSpatialAnchor.LocalAnchor = worldAnchor.GetNativeSpatialAnchorPtr();
        baseElement.UpdateWorldAnchor(worldAnchor, cloudSpatialAnchor);
    }
    else
    {
        Debug.Log("Create with a cloud anchor");
        worldAnchor.SetNativeSpatialAnchorPtr(cloudSpatialAnchor.LocalAnchor);
        baseElement.UpdateWorldAnchor(worldAnchor, cloudSpatialAnchor);
    }

    return baseElement;
}

}`

I develop in the following environment: Visual Studio 2019 enterprise 16,5 Windows 10 SDK: 10.0.18362.0 Unity2019.2.20f1 personal Azure Spatial Anchors SDK 2.2.1 HoloLens 1 OS 10.0.17763.1098

[jparismorgan]

Hi @timgoeij, thanks for letting us know. A few questions:

• Is this happening every time you run your app?
• Could you share a session id from a session that this happened during?
• Are you creating an anchor and locating anchors at the same time? The error is coming from a code path relating to locating anchors, but I also see "Creating spatial anchor" in the log.
• Could you share a full log of your session? This seems to just be partial.

This will help us investigate (internal #25956824) , thank you!

[timgoeij]

Is this happening every time you run your app? Yes

Could you share a session id from a session that this happened during? 3ed03dd0-cda3-4613-98b6-ba43969643ad

Are you creating an anchor and locating anchors at the same time? The error is coming from a code path relating to locating anchors, but I also see "Creating spatial anchor" in the log. I have similar a Debug Log when I am creating self Spatial Anchors, but this creating anchors log is comming from the Azure Spatial Anchors SDK.

Could you share a full log of your session? This seems to just be partial. See File LogFile.txt

[darax]

One thing that can lead to this nebulous 'unexpected error' is if a call in one of the Azure Spatial Anchors callbacks throws an exception. For instance, if either line here throws: WorldAnchorBackend backendAnchorForCloudAnchor = anchorsFromBackend.Find(x => x.AnchorId.Equals(Guid.Parse(args.Identifier))); CreateObject(backendAnchorForCloudAnchor, args.Anchor);

Then the the call back will see the exception and log that message.

what I think is happening is the AnchorLocated callback isn't guaranteed to be on the Unity thread. CreateObject does lots of things that can't be done off of unity's main thread. On iOS/Android the spatial anchor manager happens to put the callback on the main unity thread for other reasons. If you look at our samples we do this: protected override void OnCloudAnchorLocated(AnchorLocatedEventArgs args) { base.OnCloudAnchorLocated(args);

        if (args.Status == LocateAnchorStatus.Located)
        {
            currentCloudAnchor = args.Anchor;

            UnityDispatcher.InvokeOnAppThread(() => {the code that instantiates a prefab}

When I make an app I take it one step further and keep a ConcurrentQueue in my MonoBehavior that I process during update(). This avoids the possibilty that my class gets destroyed before the 'UnityDispatcher' gets a chance to run.

[timgoeij]

@darax thanks for the help. The AnchorsLocated callback was indeed not at the Unity thread. I solve it with the concurrentQueue

[xxia-kathy]

@darax I'm also getting the same error message but have not been able to resolve it with the concurrentQueue. Can you provide some more details on how to use it in case we are not using the concurrentQueue correctly?