search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6.01k stars 2.12k forks source link

Unity 2020.2 - OpenXR Issues - Texture coordinate channel "0" not found. #10339

Closed NLR-L closed 2 years ago

NLR-L commented 2 years ago

Describe the bug

I'm getting spammed with errors, which makes it impossible to read actual interesting errors messages. The focus provider might be broken aswell, but I'm not really using it at the moment.

I'm getting the following warning on startup:

OpenXR mesh settings are not supported; missing OpenXR extension XR_MSFT_scene_understanding_preview3
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:LogWarning(Object)
Microsoft.MixedReality.OpenXR.InternalMeshSettings:TrySetMeshComputeSettings(MeshComputeSettings)
Microsoft.MixedReality.OpenXR.MeshSettings:TrySetMeshComputeSettings(MeshComputeSettings)
Microsoft.MixedReality.Toolkit.XRSDK.OpenXR.OpenXRSpatialAwarenessMeshObserver:SetMeshComputeSettings(SpatialAwarenessMeshDisplayOptions, SpatialAwarenessMeshLevelOfDetail)
Microsoft.MixedReality.Toolkit.XRSDK.OpenXR.OpenXRSpatialAwarenessMeshObserver:ApplyUpdatedMeshDisplayOption(SpatialAwarenessMeshDisplayOptions)
Microsoft.MixedReality.Toolkit.SpatialAwareness.BaseSpatialMeshObserver:ReadProfile()
Microsoft.MixedReality.Toolkit.SpatialAwareness.BaseSpatialMeshObserver:Initialize()
Microsoft.MixedReality.Toolkit.XRSDK.GenericXRSDKSpatialMeshObserver:Initialize()
Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem:RegisterDataProvider(T)
Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem:RegisterDataProviderInternal(Boolean, Type, String, SupportedPlatforms, Object[])
Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem:RegisterDataProvider(Type, String, SupportedPlatforms, Object[])
Microsoft.MixedReality.Toolkit.SpatialAwareness.MixedRealitySpatialAwarenessSystem:InitializeInternal()
Microsoft.MixedReality.Toolkit.SpatialAwareness.MixedRealitySpatialAwarenessSystem:Initialize()
Microsoft.MixedReality.Toolkit.<>c:<InitializeAllServices>b__68_0(IMixedRealityService)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:ExecuteOnAllServicesInOrder(Action`1)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:InitializeAllServices()
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:InitializeServiceLocator()
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:RegisterInstance(MixedRealityToolkit, Boolean)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:Awake()

And after startup I'm getting the following errors every frame.

Reading UV coordinates of mesh "SpatialMesh - 4D0C3E054B598E4F-D3BF030940AC9C91" failed. Texture coordinate channel "0" not found.
UnityEngine.RaycastHit:get_textureCoord()
Microsoft.MixedReality.Toolkit.Input.MixedRealityRaycastHit:.ctor(Boolean, RaycastHit)
Microsoft.MixedReality.Toolkit.Input.DefaultRaycastProvider:Raycast(RayStep, LayerMask[], Boolean, MixedRealityRaycastHit&)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:QueryScene(IMixedRealityPointer, IMixedRealityRaycastProvider, LayerMask[], PointerHitResult, Int32, Boolean)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:UpdatePointer(PointerData)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:UpdatePointers()
Microsoft.MixedReality.Toolkit.Input.FocusProvider:Update()
Microsoft.MixedReality.Toolkit.<>c:<UpdateAllServices>b__72_0(IMixedRealityService)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:ExecuteOnAllServicesInOrder(Action`1)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:UpdateAllServices()
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:Update()

[ line -4199920]

Reading UV coordinates of mesh "SpatialMesh - 4D0C3E054B598E4F-D3BF030940AC9C91" failed. Texture coordinate channel "1" not found.
UnityEngine.RaycastHit:get_textureCoord2()
Microsoft.MixedReality.Toolkit.Input.MixedRealityRaycastHit:.ctor(Boolean, RaycastHit)
Microsoft.MixedReality.Toolkit.Input.DefaultRaycastProvider:Raycast(RayStep, LayerMask[], Boolean, MixedRealityRaycastHit&)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:QueryScene(IMixedRealityPointer, IMixedRealityRaycastProvider, LayerMask[], PointerHitResult, Int32, Boolean)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:UpdatePointer(PointerData)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:UpdatePointers()
Microsoft.MixedReality.Toolkit.Input.FocusProvider:Update()
Microsoft.MixedReality.Toolkit.<>c:<UpdateAllServices>b__72_0(IMixedRealityService)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:ExecuteOnAllServicesInOrder(Action`1)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:UpdateAllServices()
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:Update()

[ line -4199920]

To reproduce

Steps to reproduce the behavior:

I've set up and empty project and set the OpenXR settings as follows: image

And I'm using the DefaultHololens2ConfigurationProfile profile.

Your setup (please complete the following information)

Target platform (please complete the following information)

polar-kev commented 2 years ago

Hi @NLR-L, which version of the OpenXR plugin are you using? I suspect you're not on the latest version. There were also some important fixes added in the dot releases. Can you update to MRTK 2.7.2?

NLR-L commented 2 years ago

My bad, I'm using MRTK v2.7.2.0, I've downloaded from the master a week ago, which was probably commit a6471e7 since the last release had some errors in unity 2021.

As for OpenXR I'm using version 1.3.0.

image

MaxWang-MS commented 2 years ago

Hi @NLR-L, from your screenshot it seems that you are not using the latest version of Mixed Reality OpenXR plugin (under the Packages - Microsoft section). Please use the Mixed Reality Feature Tool to update it to the latest version (as of now is 1.2.0) and let us know if that solves your issue. Thanks.

NLR-L commented 2 years ago

I have updated the Mixed Reality OpenXR Plugin version to 1.2.1, and the problem still occurs.

image

This is what the diagnostic report looks like

==== Start Unity OpenXR Diagnostic Report ====

[XR] [2104] [11:04:01.463][Info   ] ==== Report Reason: System Startup Completed ====

[XR] [2104] [11:04:01.463][Info   ] ==== OpenXR Init Info ====
[XR] [2104] [11:04:01.463][Info   ] 
[XR] [2104] [11:04:01.463][Info   ] Stage 1 Missing Entry Points: (1)
[XR] [2104] [11:04:01.463][Info   ]   xrInitializeLoaderKHR
[XR] [2104] [11:04:01.463][Info   ] 
[XR] [2104] [11:04:01.464][Info   ] Stage 2 Missing Entry Points: (2)
[XR] [2104] [11:04:01.464][Info   ]   xrGetVisibilityMaskKHR
[XR] [2104] [11:04:01.464][Info   ]   xrGetD3D12GraphicsRequirementsKHR
[XR] [2104] [11:04:01.464][Info   ] 
[XR] [2104] [11:04:01.464][Info   ] ==== OpenXR Provider Info ====
[XR] [2104] [11:04:01.464][Info   ] 
[XR] [2104] [11:04:01.464][Info   ] Spec Version: 1.0.17
[XR] [2104] [11:04:01.464][Info   ] Provider Version: 1.3.1
[XR] [2104] [11:04:01.464][Info   ] App:  #3159629941
[XR] [2104] [11:04:01.464][Info   ] Engine: 2021.2.4f1 #35668993
[XR] [2104] [11:04:01.464][Info   ] 
[XR] [2104] [11:04:01.464][Info   ] ==== OpenXR Runtime Info ====
[XR] [2104] [11:04:01.464][Info   ] 
[XR] [2104] [11:04:01.464][Info   ] Features requested to be enabled: (5)
[XR] [2104] [11:04:01.465][Info   ]   Eye Gaze Interaction Profile: Version=0.0.1, Company="Unity", Extensions="XR_EXT_eye_gaze_interaction"
[XR] [2104] [11:04:01.465][Info   ]   Hand Tracking: Version=1.2.1, Company="Microsoft", Extensions="XR_EXT_hand_tracking XR_MSFT_hand_tracking_mesh"
[XR] [2104] [11:04:01.465][Info   ]   Microsoft Hand Interaction Profile: Version=0.0.1, Company="Unity", Extensions="XR_MSFT_hand_interaction"
[XR] [2104] [11:04:01.465][Info   ]   Mixed Reality Features: Version=1.2.1, Company="Microsoft", Extensions="XR_MSFT_holographic_window_attachment XR_MSFT_unbounded_reference_space XR_MSFT_spatial_anchor XR_MSFT_secondary_view_configuration XR_MSFT_first_person_observer XR_MSFT_spatial_graph_bridge XR_MSFT_perception_anchor_interop XR_MSFT_spatial_anchor_persistence XR_MSFT_scene_understanding XR_MSFT_scene_understanding_serialization XR_MSFT_spatial_anchor_export_preview XR_MSFT_composition_layer_reprojection"
[XR] [2104] [11:04:01.465][Info   ]   Motion Controller Model: Version=1.2.1, Company="Microsoft", Extensions="XR_MSFT_controller_model"
[XR] [2104] [11:04:01.465][Info   ] 
[XR] [2104] [11:04:01.465][Info   ] Requested feature extensions not supported by runtime: (3)
[XR] [2104] [11:04:01.465][Info   ]   XR_MSFT_spatial_anchor_persistence: Feature="Mixed Reality Features": Version=1.2.1, Company="Microsoft"
[XR] [2104] [11:04:01.465][Info   ]   XR_MSFT_spatial_anchor_export_preview: Feature="Mixed Reality Features": Version=1.2.1, Company="Microsoft"
[XR] [2104] [11:04:01.465][Info   ]   XR_MSFT_controller_model: Feature="Motion Controller Model": Version=1.2.1, Company="Microsoft"
[XR] [2104] [11:04:01.465][Info   ] 
[XR] [2104] [11:04:01.465][Info   ] Available Runtime Extensions: (22)
[XR] [2104] [11:04:01.465][Info   ]   XR_KHR_win32_convert_performance_counter_time: Version=1
[XR] [2104] [11:04:01.465][Info   ]   XR_KHR_D3D11_enable: Version=5
[XR] [2104] [11:04:01.465][Info   ]   XR_KHR_D3D12_enable: Version=7
[XR] [2104] [11:04:01.465][Info   ]   XR_KHR_composition_layer_depth: Version=5
[XR] [2104] [11:04:01.466][Info   ]   XR_KHR_composition_layer_color_scale_bias: Version=5
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_unbounded_reference_space: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_spatial_anchor: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_perception_anchor_interop: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_EXT_win32_appcontainer_compatible: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_EXT_conformance_automation: Version=3
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_composition_layer_reprojection: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_spatial_graph_bridge: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_EXT_eye_gaze_interaction: Version=1
[XR] [2104] [11:04:01.466][Info   ]   XR_MSFT_hand_interaction: Version=1
[XR] [2104] [11:04:01.467][Info   ]   XR_EXT_hand_tracking: Version=4
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_hand_tracking_mesh: Version=3
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_holographic_window_attachment: Version=1
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_secondary_view_configuration: Version=1
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_first_person_observer: Version=1
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_scene_understanding: Version=1
[XR] [2104] [11:04:01.467][Info   ]   XR_MSFT_scene_understanding_serialization: Version=1
[XR] [2104] [11:04:01.468][Info   ]   XR_EXT_debug_utils: Version=4
[XR] [2104] [11:04:01.468][Info   ] 
[XR] [2104] [11:04:01.468][Info   ] Available Layers: (0)
[XR] [2104] [11:04:01.470][Info   ] 
[XR] [2104] [11:04:01.470][Info   ] Runtime Name: Windows Mixed Reality Runtime
[XR] [2104] [11:04:01.470][Info   ] Runtime Version: 107.2109.10010
[XR] [2104] [11:04:01.470][Info   ] 
[XR] [2104] [11:04:01.471][Info   ] Form factor: XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY
[XR] [2104] [11:04:01.471][Info   ] 
[XR] [2104] [11:04:01.471][Info   ] Runtime extensions enabled: (17)
[XR] [2104] [11:04:01.471][Info   ]   XR_EXT_eye_gaze_interaction: Version=1
[XR] [2104] [11:04:01.471][Info   ]   XR_EXT_hand_tracking: Version=4
[XR] [2104] [11:04:01.471][Info   ]   XR_KHR_D3D11_enable: Version=5
[XR] [2104] [11:04:01.471][Info   ]   XR_KHR_composition_layer_depth: Version=5
[XR] [2104] [11:04:01.471][Info   ]   XR_KHR_win32_convert_performance_counter_time: Version=1
[XR] [2104] [11:04:01.471][Info   ]   XR_MSFT_composition_layer_reprojection: Version=1
[XR] [2104] [11:04:01.471][Info   ]   XR_MSFT_first_person_observer: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_hand_interaction: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_hand_tracking_mesh: Version=3
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_holographic_window_attachment: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_perception_anchor_interop: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_scene_understanding: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_scene_understanding_serialization: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_secondary_view_configuration: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_spatial_anchor: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_spatial_graph_bridge: Version=1
[XR] [2104] [11:04:01.472][Info   ]   XR_MSFT_unbounded_reference_space: Version=1
[XR] [2104] [11:04:01.472][Info   ] 
[XR] [2104] [11:04:01.472][Info   ] Available Reference Spaces: (3)
[XR] [2104] [11:04:01.472][Info   ]   XR_REFERENCE_SPACE_TYPE_VIEW
[XR] [2104] [11:04:01.472][Info   ]   XR_REFERENCE_SPACE_TYPE_LOCAL
[XR] [2104] [11:04:01.472][Info   ]   XR_REFERENCE_SPACE_TYPE_UNBOUNDED_MSFT
[XR] [2104] [11:04:01.472][Info   ] 
[XR] [2104] [11:04:01.472][Info   ] Instance Id  : 0x1
[XR] [2104] [11:04:01.472][Info   ] System Id    : 0x1
[XR] [2104] [11:04:01.473][Info   ] Session Id   : 0x2
[XR] [2104] [11:04:01.473][Info   ] 
[XR] [2104] [11:04:01.473][Info   ] Available View Configuration Types: (2)
[XR] [2104] [11:04:01.473][Info   ]   XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO (Selected): FovMutable=True
[XR] [2104] [11:04:01.473][Info   ]     [0]: Recommended: Width=1440, Height=936, SampleCount=1
[XR] [2104] [11:04:01.473][Info   ]     [0]:     Maximum: Width=2880, Height=1872, SampleCount=4
[XR] [2104] [11:04:01.473][Info   ]     [1]: Recommended: Width=1440, Height=936, SampleCount=1
[XR] [2104] [11:04:01.473][Info   ]     [1]:     Maximum: Width=2880, Height=1872, SampleCount=4
[XR] [2104] [11:04:01.473][Info   ]   XR_VIEW_CONFIGURATION_TYPE_SECONDARY_MONO_FIRST_PERSON_OBSERVER_MSFT: FovMutable=True
[XR] [2104] [11:04:01.473][Info   ]     [0]: Recommended: Width=2272, Height=1276, SampleCount=1
[XR] [2104] [11:04:01.473][Info   ]     [0]:     Maximum: Width=4544, Height=2556, SampleCount=4
[XR] [2104] [11:04:01.473][Info   ] 
[XR] [2104] [11:04:01.473][Info   ] Skip Present To Main Screen: Yes
[XR] [2104] [11:04:01.473][Info   ] 
[XR] [2104] [11:04:01.473][Info   ] Available Environment Blend Modes: (1)
[XR] [2104] [11:04:01.473][Info   ]   XR_ENVIRONMENT_BLEND_MODE_ADDITIVE (Selected)
[XR] [2104] [11:04:01.473][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] System Properties: Name="Windows Mixed Reality", VendorId=1118
[XR] [2104] [11:04:01.474][Info   ] System Graphics Properties: MaxWidth=16384, MaxHeight=16384, MaxLayers=16
[XR] [2104] [11:04:01.474][Info   ] System Tracking Properties: OrientationTracking=True, PositionTracking=True
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] ==== OpenXR Input ====
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] Action Sets (2): 
[XR] [2104] [11:04:01.474][Info   ]   eyegaze: ActionCount=1
[XR] [2104] [11:04:01.474][Info   ]   microsofthandinteraction: ActionCount=6
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] ==== OpenXR Swapchain Details ====
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] Render Mode: Single Pass Instanced
[XR] [2104] [11:04:01.474][Info   ] Depth Submission Mode: Depth16Bit
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] Swapchain Formats:  [c:91] 87 28 29 [d:55] 45 40 20
[XR] [2104] [11:04:01.474][Info   ] Swapchain Views: (2)
[XR] [2104] [11:04:01.474][Info   ]   [0]: Width=1440, Height=936, SampleCount=1
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] ==== Last 20 non-XR_SUCCESS returns ====
[XR] [2104] [11:04:01.474][Info   ] [SUCCESS] xrGetReferenceSpaceBoundsRect: XR_SPACE_BOUNDS_UNAVAILABLE (2x)
[XR] [2104] [11:04:01.474][Info   ] 
[XR] [2104] [11:04:01.474][Info   ] ==== Last 20 Events ====
[XR] [2104] [11:04:01.474][Info   ] XrEventDataSessionStateChanged: XR_SESSION_STATE_UNKNOWN->XR_SESSION_STATE_IDLE
[XR] [2104] [11:04:01.474][Info   ] XrEventDataSessionStateChanged: XR_SESSION_STATE_IDLE->XR_SESSION_STATE_READY
[XR] [2104] [11:04:01.474][Info   ] XrEventDataSessionStateChanged: XR_SESSION_STATE_READY->XR_SESSION_STATE_SYNCHRONIZED
[XR] [2104] [11:04:01.474][Info   ] XrEventDataSessionStateChanged: XR_SESSION_STATE_SYNCHRONIZED->XR_SESSION_STATE_VISIBLE
[XR] [2104] [11:04:01.474][Info   ] XrEventDataSessionStateChanged: XR_SESSION_STATE_VISIBLE->XR_SESSION_STATE_FOCUSED
[XR] [2104] [11:04:01.475][Info   ] 

==== End Unity OpenXR Diagnostic Report ====
NLR-L commented 2 years ago

I think the issue stems from MixedRealityRaycastHit not checking if the mesh has a texture before writing the texcoords to the RaycastHit struct. I don't think the HoloLens spatial mesh has uv's. So perhaps there could be a check before trying to attempt to get the textcoords.

image Texcoords makes a call using the (probably non-existent) texture channels.

NLR-P commented 2 years ago

There was a problem in MixedRealityRaycastHit.cs , this assumed a UV was present in all hits it found. That caused the error. As Hololens doesn't create UV's , there was no way the raycast hit could set it's TextureCoord or its lightmapCoord. I changed the code to below, now no more errors: 

public MixedRealityRaycastHit(bool raycastValid, RaycastHit hitInfo)
        {
            this.raycastValid = raycastValid;
            if (raycastValid)
            {
                point = hitInfo.point;
                normal = hitInfo.normal;
                barycentricCoordinate = hitInfo.barycentricCoordinate;
                distance = hitInfo.distance;
                triangleIndex = hitInfo.triangleIndex;

                MeshCollider meshCollider = hitInfo.collider as MeshCollider;

                if (meshCollider?.sharedMesh?.uv?.Length > 0)
                {
                    textureCoord = hitInfo.textureCoord;

                    textureCoord2 = hitInfo.textureCoord2;

                    lightmapCoord = hitInfo.lightmapCoord;
                }
                else
                {
                    textureCoord = Vector2.zero;
                    textureCoord2 = Vector2.zero;
                    lightmapCoord = Vector2.zero;
                }

                transform = hitInfo.transform;
                collider = hitInfo.collider;
            }
            else
            {
                point = Vector3.zero;
                normal = Vector3.zero;
                barycentricCoordinate = Vector3.zero;
                distance = 0;
                triangleIndex = 0;
                textureCoord = Vector2.zero;
                textureCoord2 = Vector2.zero;
                transform = null;
                lightmapCoord = Vector2.zero;
                collider = null;
            }
        }
keveleigh commented 2 years ago

@NLR-P Thanks for the suggestion above! I ended up taking a similar path, but instead used the HasVertexAttribute method to query each index individually. I'm no longer seeing the logs! The fix is open for 2.7.3 in #10370

keveleigh commented 2 years ago

This should be fixed via #10370

NLR-L commented 2 years ago

Nice work @keveleigh 🙂