search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

Invoking SkinnedMeshRenderer.BakeMesh() with tracked hand mesh in HoloLens 2 is creating an invalid mesh with OpenXR. #11378

Closed krishx007 closed 9 months ago

krishx007 commented 1 year ago

Describe the bug

Invoking SkinnedMeshRenderer.BakeMesh() with tracked hand mesh in HoloLens 2 is creating a mesh with its default mesh geometry rather than a mesh which represent the current deformation of the hand mesh. The generated mesh also not at its intended position, it seems to spawn at world origin or head position. We are facing this issue after transitioning our project to use MRTK OpenXR. (Mixed Reality OpenXR Plugin v1.7) It is happening with latest MRTK version 2.8.3 and with Unity 2020.3.43

To reproduce

Steps to reproduce the behavior: 1)Open the "BasicSample" unity project with latest version of Unity 2020. (We used Unity 2020.3.43) https://github.com/microsoft/OpenXR-Unity-MixedReality-Samples/tree/main/BasicSample

2) Upgrade the MRTK (Foundation & StandardAssets packages) and Mixreality OpenXR plugins to 2.8.3

3)Now attach some script to "RiggedHandLeft" or "RiggedHandRight" prefab which will capture the mesh of SkinnedMeshRenderer component associated with the tracked hand prefab.

You can try using this mono script, just attached it to "RiggedHandLeft" or "RiggedHandRight" prefab and trigger "BakeMesh()" when hand is tracking in HoloLens 2. https://gist.github.com/krishx007/a645f77c1284b2f69479da8e3266fce1

Expected behavior

Invoking SkinnedMeshRenderer.BakeMesh() with tracked hand mesh in HoloLens 2 should generate a mesh geometry similar to the current deformation of the hand mesh at the exact same position where the tracked hand mesh was visible.

Screenshots

If applicable, add screenshots to help explain your problem.

Your setup (please complete the following information)

Unity 2020.3.43 MRTK version 2.8.3 Mixreality OpenXR plugins to 2.8.3

Target platform (please complete the following information)

Additional context

Add any other context about the problem here.

krishx007 commented 1 year ago

Hi @keveleigh, could you please have look at this bug. Our project progress has been stalled because of this bug.

DenisTschitschenkow commented 1 year ago

11399 is describing the root of the issue, with a work around.

marlenaklein-msft commented 1 year ago

@krishx007 Were you able to resolve this issue?

IssueSyncBot commented 9 months ago

Hi @krishx007,

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visithttps://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!