microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

>= Unity 2020.1 compatible persistent, local WorldAnchorStore #8796

Closed tmhab closed 9 months ago

tmhab commented 3 years ago

Describe the problem

Unity 2020.1 will remove WorldAnchorStore functionality.

Describe the solution you'd like

WorldAnchorManager wrapping another solution that allows for local, persistent world anchors.

Describe alternatives you've considered

Additional context

N/A

keveleigh commented 3 years ago

This is coming! We've been working with Unity to add an extension to the Windows XR Plugin to allow for anchor store access for XRAnchor/XRReferencePoint.

tmhab commented 3 years ago

Fantastic; good to hear! I'll be looking forward to seeing it implemented 👍

keveleigh commented 3 years ago

Finally an update! The API has shipped (or will ship any moment) in the 2.5 package.

The APIs to use are TryGetAnchorStoreAsync and the resulting XRAnchorStore APIs.

Happy to provide some additional examples of use if requested, and we should update WorldAnchorStore in MRTK to support this API.

wspaas commented 3 years ago

@keveleigh I am trying to write an implementation that uses the new API for placing, saving and sharing Spatial Anchors.

I got this to work using the XRReferencePointSubsystem and XRAnchorStore. My Code is based on an example found here:
https://forum.unity.com/threads/world-anchors-on-the-hl2-with-the-new-xr-plugin.1035256/#post-6711919

But currently my code only allows me to persistently save anchors on 1 device (local). I would like to share the saved Anchor to another device. Only Saving the TrackableId seems to be insufficient to 'find' the Anchor when not saved to the XRAnchorStore (which is only local data, I assume).

What is the best way to share the locations of these anchors between devices (without using Azure)?

I found this class in the native UWP code:
https://docs.microsoft.com/en-us/uwp/api/windows.perception.spatial.spatialanchortransfermanager?view=winrt-19041

Is there an already existing implementation of this within the Unity API? Or some examples of how to best combine this with the existing API? Or maybe there is an entirely different way which I overlooked?

Thanks in advance!

NoTuxNoBux commented 3 years ago

I think before there was the WorldAnchorTransferBatch in C# as documented by this documentation page, but maybe this is for the deprecated API and an equivalent doesn't exist yet in the new API?

tmhab commented 3 years ago

@wspaas I'm very interested in your findings when you hopefully find a solution, please update us if you make progress :)

zantiu commented 3 years ago

Any update on this?

Axiang666 commented 3 years ago

Looking forward to the update

Kent1LG commented 3 years ago

Hello, my team and I also really need a solution on this when it's possible ! Any news?

NLR-L commented 3 years ago

Hi, are there any updates?

shoukailiang commented 3 years ago

Hello,I also need a solution.

JulieMo3 commented 2 years ago

Still looking for a solution. Any news?

NLR-L commented 2 years ago

I'm sorry I can't post the complete solution.

But you can make use of the unity ArFoundation package, to convert an anchor to an anchor pointer. This pointer can be converted to the Windows.Perception.Spatial.SpatialAnchor.

How you can do this can be found in this thread. When using OpenXR as the XR plugin you need to do the extra step that's done in OP's last post. https://forum.unity.com/threads/arfoundation-aranchor-nativeptr-needs-extra-conversion-for-openxr.1199446/

This pointer works with OpenXR but also with the WindowsMixedReality XR plugin.

Then you can use this SpatialAnchor together with the SpatialAnchorExporter to export your anchors.

image

When you read out this data you can import it with the SpatialAnchorTransferManager.

image

The position of the anchor can be calculated by combining the code from within the Azure Spatial Anchor Manager with the Unity Spatial Coordinate system.

SpatialCoordinateSystem unityWorldCoordinateSystem = PerceptionInterop.GetSceneCoordinateSystem(Pose.identity) as SpatialCoordinateSystem;
SpatialCoordinateSystem anchorCoordinateSystem = anchor.CoordinateSystem;
System.Numerics.Matrix4x4? anchorToUnityWorld = anchorCoordinateSystem.TryGetTransformTo(unityWorldCoordinateSystem);
if (anchorToUnityWorld.HasValue)
{
    System.Numerics.Quaternion anchorToUnityWorldRotationNative = System.Numerics.Quaternion.CreateFromRotationMatrix(anchorToUnityWorld.Value);

    // The anchor world rotation
    Quaternion anchorToUnityWorldRotation = ToUnityQuaternion(anchorToUnityWorldRotationNative);

    // The anchor world position
    Vector3anchorToUnityWorldTranslation = ToUnityVector3(anchorToUnityWorld.Value.Translation);
} 
else 
{
    throw new InvalidOperationException("Couldn't compute a native anchor pose");
}

This way you can import/export anchors without having to use Azure.

keveleigh commented 2 years ago

For OpenXR, we have some samples showing off the new XRAnchorStore APIs: https://github.com/microsoft/OpenXR-Unity-MixedReality-Samples/blob/3917606bae2dd495cdd5680d33701ccd2caee3d7/BasicSample/Assets/ARAnchor/Scripts/AnchorsSample.cs#L51

Morgan-6Freedom commented 12 months ago

What's the status ? What is the solution to share a world anchor locally between 2 HL2 ?

IssueSyncBot commented 9 months ago

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all-new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visit https://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!