search

Azure / azure-spatial-anchors-samples

Azure Spatial Anchors sample code
Other
293 stars 138 forks source link

Multi-user capabilities - coordinate system of the different players dosen't line up #403

Closed mikkeline-elleby closed 1 year ago

mikkeline-elleby commented 1 year ago

Hi,

I'm using unity 2020.3.42f1, I'm running it with a PUN app and have 2 different hololens where I deploy the application (+my computer)

I'm doing my bachelor project on the multi-user capabilities of the Hololens. I have tried to follow this tutorial: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/tutorials/mr-learning-sharing-01

Everything is working fine with the pun app, and I am able to view the players inside of the Room. The only thing missing for making it a shared experience is to have the coordinate system of all the different players to be the same. I have tried to introduce an azure spatial anchor to to share the location of the table (the object in the SharedPlayGround that defines the coordinate system of the position of the players) to the players so that their coordinate system lines up. The anchor is shared with the Sharing Module script and the Anchor Module. But whenever I deploy the application on two different hololens the position dosen't match up. (the player is also represented with a sphere and it dosen't coincide with the real position of the player.

How to run the app: wearer A:

wearer B:

20230426_103216_HoloLens

20230426_103315_HoloLens

I don't know if the null reference is the root of my issue. Another idea I have, is that maybe the environment isn't scanned enough before creating the achor (but I have a log telling me if that was the case - and it dosen't get printed so are we fine ?)

It would help me so much if you could help me, I have been struggling with this problems for weeks.

darax commented 1 year ago

Hi @mikkeline-elleby Are you rendering something (like a cube) where the anchor is being detected on both devices? That null reference might be photon trying to spawn an object, so it would be good to figure it out. You can get a full log in the device portal in the application's 'TempState' folder.

mikkeline-elleby commented 1 year ago

(Just the cube is the table anchor that I changed the dimension of so I could see what happened when I moved it, but it the setup from the tutorial.)

Here's the log :

UnityPlayer (2).log

(I ran the app 2 times and here I don't get the starting NullReferenceException: UnityPlayer (1).log )

I tried to understand it to the best of my capabilities. I found three NullReferenceException and I think that they are linked. Here when I try to share my local anchor with the sharingModulScript from the multiUsercapabilities asset it is not correctly linked to an object. Which I do not really understand because it seems to have an object ? And the when I try to get the anchor position I get this, so I can see that I can not get the position of the object but this is weird as I print the position and the rotation of the anchor to my Log and it gives me coordinates sure enough.

NullReferenceException: Object reference not set to an instance of an object. at MRTK.Tutorials.MultiUserCapabilities.SharingModuleScript.ShareAzureAnchor () [0x00000] in <00000000000000000000000000000000>:0

NullReferenceException: Object reference not set to an instance of an object. at UnityEngine.Component.get_transform () [0x00000] in <00000000000000000000000000000000>:0 at UnityEngine.XR.ARFoundation.ARTrackableManager`5

mikkeline-elleby commented 1 year ago

I just put my project on a repo so you can have a look at it : https://github.com/mikkeline-elleby/multiUser-Capabilities

Could also the issue be that my anchor is not on the surface of a table or the floor ? So that the position is not correctly set and therefore my spatial anchor has problems when being set on someone elses device ? Or does that not matter? )

(and also I was just wondering if I'm just starting out with the hololens, and I spend so much time on this project - can I get it to work ? As this is fundemental for all of the rest of my project)

darax commented 1 year ago

I'm not seeing your repo. The link seems to link back here.

You don't need to put the anchors on surfaces to get a multi user experience to work. It does look like something in 'AnchorModuleScript' is causing an issue.

I'm somewhat concerned about entries like:

Unable to register IMixedRealitySpatialAwarenessObserver data provider (Windows Mixed Reality Spatial Mesh Observer) because the value of concreteType is null.
This may be caused by code being stripped during linking. The link.xml file in the MixedRealityToolkit.Generated folder is used to control code preservation.
More information can be found at https://docs.unity3d.com/Manual/ManagedCodeStripping.html.
mikkeline-elleby commented 1 year ago

I updated the link, which should work now (I hope) : https://github.com/mikkeline-elleby/multiUser-Capabilities

And my stripping is Low in my project settings so this is a bit weird.

darax commented 1 year ago

https://github.com/mikkeline-elleby/multiUser-Capabilities/blob/f652afa2f128e0742e131977ff9f2b47e8839f69/Assets/MRTK.Tutorials.AzureSpatialAnchors/Scripts/AnchorModuleScript.cs#L405 that second call to CreateNativeAnchor shouldn't be made. In fact, I'm not sure the first should be made. The basic demo (https://github.com/Azure/azure-spatial-anchors-samples/blob/de2d9db08a83cf736b137050c7dbec0739529334/Unity/Assets/AzureSpatialAnchors.Examples/Scripts/AzureSpatialAnchorsBasicDemoScript.cs#L99) has a simpler approach. Creating the game object looks more like: https://github.com/Azure/azure-spatial-anchors-samples/blob/de2d9db08a83cf736b137050c7dbec0739529334/Unity/Assets/AzureSpatialAnchors.Examples/Scripts/DemoScriptBase.cs#L503

mikkeline-elleby commented 1 year ago

Hi I have tested your idea and tried different things but none worked.

Maybe it's that the anchors is not being kept correctly in memory, or maybe it's the sharing bit as I get the NullPointerException when I want to share it and when I try to get it back. I can see also that the feedback getting printed to the log that the ID is getting shared and created could be wrong according to this : https://learn.microsoft.com/en-us/answers/questions/850617/(unity-hololens2)-spatial-anchor-id-could-not-be-i

What should I try to do ? Is there another ready to use sharing module for azure anchors I could try out ? Or do you see other issues I could try to corrrect ?

mikkeline-elleby commented 1 year ago

I found a solution @darax. The error was due to the Table Anchor. I used this tutorial: https://learn.microsoft.com/en-us/training/modules/azure-spatial-anchors-tutorials/3-exercise-get-started-with-azure-spatial-anchors and implemented photon on it and adding the sharing module from the multi-user capabibilities. Here is my repo : https://github.com/mikkeline-elleby/Multi-user-Hololens-PUN-and-ASA I have tested it out and it works really great (even with 3 hololens). I will write a report on everything as this is my bachelor project. If you need anymore details or a step-by-step of how I solved the issue then just reach out (I can also help update the tutorial if needed)

darax commented 1 year ago

Hey @mikkeline-elleby , I'm glad to hear you figured it out. I wish I had the time to have provided more guidance. Back in the day getting multiple users in an AR/VR environment was my favorite thing to do. :) https://learn.microsoft.com/en-us/windows/mixed-reality/deprecated/mixed-reality-250.