search

MixedRealityToolkit / MixedRealityToolkit-Unity

This repository holds the third generation of the Mixed Reality Toolkit for Unity. The latest version of the MRTK can be found here.
BSD 3-Clause "New" or "Revised" License
358 stars 91 forks source link

[BUG] Controllers appearing in HoloLens 2 for both hands and hand interaction not working. #491

Open farhadpiri opened 10 months ago

farhadpiri commented 10 months ago

Describe the bug

A clear and concise description of what the bug is. I cloned the project at https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity and I installed Unity 2021.3.21f1 to exactly match the project in details.

To reproduce

Steps to reproduce the behavior:

  1. I cloned the project at https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
  2. I installed Unity 2021.3.21f1 to exactly match the project in details.
  3. Go to build settings add Open XR and Windows MR feature group. 4.Make sure HoloLens remote feature group is also checked.
  4. Make sure Eye Gaze and Microsoft Hand interaction is within profilers.
  5. Open "Hand Interaction Example "scene, then from Mixed reality enable remote play mode and enter Ip address
  6. Confirm controllers appearing in remote player for both hands
  7. Confirm hand interaction not working with no visible poke interactor either, but hand ray is working for both the ray and air tap or

Expected behavior

Hand interactions work with poke interactor and controllers should not be shown.

Screenshots

image

image

image

Your setup (please complete the following information)

Additional context

Here is the video captured through MRC: https://drive.google.com/file/d/1OwPS9L2_l4vEDNbsVYgoTHucFtKqPn2u/view?usp=sharing

AMollis commented 10 months ago

@farhadpiri we cannot reproduce this issue.

Your screenshot shows Win32/Stand-Alone app settings. For HoloLens you need to set the "UWP OpenXR" settings (Windows Icon). Under the "UWP" OpenXR settings please ensure you have the Hand interaction profile and the Hand Tracking enabled. (Also, for the best performance, we recommend app's use 16-bit depth if you are not needing the stencil buffer)

Image

AMollis commented 10 months ago

@farhadpiri , can you please answer the following

  1. Is this 100% reproducible for you?
  2. It appears you are using app remoting. Please try deploying to HoloLens 2 without remoting, does this repro without remoting? Also
  3. Please update to Microsoft's Mixed Reality OpenXR 1.9 plugin. and try again. Does it repro with 1.9?
  4. What version of the OS are you using?
  5. Are there any errors in the app's Unity log file?
farhadpiri commented 10 months ago

@AMollis Sure, anytime BW, I needed Desktop for Playmode though. I also checked those boxes for UWP and nothing changed.

  1. Yes, this is 100% reproducable for me in any project using holographic remote in playmode.
  2. When deployed, controllers don`t show up and poke interactor works fine.
  3. Yes Microsoft's Mixed Reality OpenXR 1.9 plugin is replaced with the 1.7 and this issue still persists.
  4. I am using 21H1 with Build Number 20348.1432 and the reason for downgrade was that research mode has issues when you access PV data together with other sensors in latest OS build. However, controllers appearing was still the case with the updated windows as well. Referring to the related issue here: https://stackoverflow.com/questions/73440844/reading-pv-and-depth-simultaneously-in-hololens
  5. As you know the app is working fine when packaged, unlike the Playmode.

Additional Context: I also have a personal project derived from MRTKDevTemplate that is published and shows different behaviors now and then. Even though the app is published and nothing changes on app launch, sometimes you see controllers appearing and sometimes you do not. Clean build fixes this issue for a while and then it appears again after a period of time without any new app deployment. I am planning to open another issue for this though. I also worked a lot with XR initialization setups and other possible solutions to fix it, but, all to no avail. That is why I didn`t mention this at the beginning.

farhadpiri commented 10 months ago

May be I am facing with some Hardware Failure issues?!

marlenaklein-msft commented 10 months ago

@farhadpiri Thanks for confirming your setup details. I still have not been able to get a consistent repo on this. Are there any errors or warnings logged in the console? We've observed the controller appearing in the past very inconsistently. We'll use this bug to track progress on resolving the issue.

AMollis commented 7 months ago

Hi @farhadpiri ,

Thank you for reporting this issue or requesting this feature. We appreciate your feedback and interest in MRTK3.

We have reviewed your issue or feature request and confirmed that it is valid. We have added it to our backlog and will prioritize it accordingly.

However, if you would like to speed up the process and contribute to the MRTK3 project, you are welcome to submit a Pull Request with your proposed fix or implementation. We would be happy to review your code and merge it into the main branch if it meets our quality standards.

Please let us know if you have any questions or need any assistance. Thank you for your support and collaboration.

Best regards, The MRTK3 team

marlenaklein-msft commented 7 months ago

@keveleigh Had you ever looked into this issue?

ProgBloger commented 7 months ago

I have this issue as well. Is there a fix for it?