Scene Understanding crashes on awaiter.GetResult(); during Holographic Remoting with MRTK 2.7.3

[Asif-Zahir]

Describe the bug

I tried to test the code snippets according to documentation: https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unity/scene-understanding-sdk. I am unable to run my project in playmode/Holographic Remoting due to this line:

Scene myScene = SceneObserver.ComputeAsync(querySettings, 10.0f).GetAwaiter().GetResult();

Unity get stuck on Application.EnterPlayMode.

To reproduce

Steps to reproduce the behavior:

1) Open the Scene Understanding Example Scene 2) Add an empty gameobject 3) Add a script with the following:

    async void Start()
    {
        if (!SceneObserver.IsSupported())
        {
            Debug.LogWarning("Huh. SceneObserver isn't supported.");
        }
        Debug.LogWarning("Initializing scene understanding");

        await InitAsync();
    }

    public async Task InitAsync()
    {
        SceneObserverAccessStatus accessStatus = await SceneObserver.RequestAccessAsync();
        Debug.LogWarning($"Access status: {accessStatus}");

        // Create Query settings for the scene update
        SceneQuerySettings querySettings;

        querySettings.EnableSceneObjectQuads = true;                                       // Requests that the scene updates quads.
        querySettings.EnableSceneObjectMeshes = true;                                      // Requests that the scene updates watertight mesh data.
        querySettings.EnableOnlyObservedSceneObjects = false;                              // Do not explicitly turn off quad inference.
        querySettings.EnableWorldMesh = true;                                              // Requests a static version of the spatial mapping mesh.
        querySettings.RequestedMeshLevelOfDetail = SceneMeshLevelOfDetail.Medium;            // Requests the finest LOD of the static spatial mapping mesh.

        // Initialize a new Scene
        var x = SceneObserver.ComputeAsync(querySettings, 10.0f);
        var awaiter = SceneObserver.ComputeAsync(querySettings, 10.0f).GetAwaiter();

        Debug.LogWarning("Compute Async");

        Scene myScene = awaiter.GetResult();

        Debug.LogWarning("Got Result");

    }

Expected behavior

I was hoping the project to run and to see the warning log "Got Result".

Current behavior

Project does not compile, unity freezes as it tries to load play mode/remoting - stuck on Application.EnterPlayMode

Your setup (please complete the following information)

• Unity Version 2020.3.15f2
• MRTK Version 2.7.3

Target platform (please complete the following information)

• HoloLens 2

[MaxWang-MS]

Hi @Asif-Zahir, could you try running the sample scene based on the instructions here? The doc you linked is more for people not using MRTK. MRTK already handles the part you mentioned for you.

[Asif-Zahir]

@MaxWang-MS this works. However, I was hoping to write a script on top of this to instantiate objects on a floor for example. How would I go about doing this?

[MaxWang-MS]

@Asif-Zahir I would say take a look at the following class and function:https://github.com/microsoft/MixedRealityToolkit-Unity/blob/685a8f5daf6426fae9d73cde6af9894f692d8886/Assets/MRTK/Examples/Experimental/SceneUnderstanding/Scripts/DemoSceneUnderstandingController.cs#L104 You should be able to modify that function directly or create a similar class to achieve your goal.

[IssueSyncBot]

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all-new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visit https://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!