[BUG] Far ray origin/aim rotating with camera when using handtracking.

[TheXRMonk]

Describe the bug

Far ray origin rotating with camera when using handtracking.

To reproduce

Steps to reproduce the behavior:

Use HandInteractionExamples scene with Oculus link (also broken in builds). Simply observe how the ray's aim is influenced by headset rotation.

Expected behavior

Only the tracked hand should have an influence on far ray aim.

Screenshots

(Cube added for visualizing headset pose)

Your setup (please complete the following information)

• Unity Version 2022.3.12f1
• Commit 29a5fa5 - Update README.md (#11775)

Target platform (please complete the following information)

• OpenXR
• Meta Quest 3

[AMollis]

@TheXRMonk , if you have a purposed fix for Meta Quest 3, please make a PR and we will review. We encourage the community to help address these Meta Quest issues.

[TheXRMonk]

@TheXRMonk , if you have a purposed fix for Meta Quest 3, please make a PR and we will review. We encourage the community to help address these Meta Quest issues.

I don't have a fix for this. And i don't think it's quest 3 related (since it's also appearing directly in the editor through link, and on MagicLeap 2 builds with handtracking).

[whebertML]

@TheXRMonk Can you elaborate on what you are seeing on MagicLeap2 with this issue? Are you seeing it when using the MagicLeap App Simulator to link the device to the Unity Editor Play session? Or are you seeing an issue when deployed to device?

The MRTK3 algorithm for hand ray does cause the ray to be slightly influenced with head rotation, but perhaps not as much as your screen capture above shows.

[TheXRMonk]

@TheXRMonk Can you elaborate on what you are seeing on MagicLeap2 with this issue? Are you seeing it when using the MagicLeap App Simulator to link the device to the Unity Editor Play session? Or are you seeing an issue when deployed to device?

The MRTK3 algorithm for hand ray does cause the ray to be slightly influenced with head rotation, but perhaps not as much as your screen capture above shows.

The capture above is in the editor using Oculus Link - not MagicLeap2. Are there any settings I can try adjusting for the "head rotation aim"-multiplier?

[whebertML]

@TheXRMonk Understood. I have a Quest3 and was going to try and reproduce your issue with link when time permits. But you also mentioned you are seeing the issue on MagicLeap2? Does it appear exactly the same on MagicLeap2 as it does in Editor via Quest link for you, or are there differences? Trying to determine if it is just the issue of the ray being influenced by headpose rotation, or something more severe in your Editor via Quest link use case above.

Regarding the algorithm, looks to be using hard-coded constants, so not adjustable in settings. But you could possibly alter locally to see if it fixes anything for you: https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/blob/main/org.mixedrealitytoolkit.input/Utilities/HandRay.cs#L73

[TheXRMonk]

@TheXRMonk Understood. I have a Quest3 and was going to try and reproduce your issue with link when time permits. But you also mentioned you are seeing the issue on MagicLeap2? Does it appear exactly the same on MagicLeap2 as it does in Editor via Quest link for you, or are there differences? Trying to determine if it is just the issue of the ray being influenced by headpose rotation, or something more severe in your Editor via Quest link use case above.

Exactly, I built it to MagicLeap and Quest3 just to confirm that the behaviour is not platform/build specific (We first noticed it in the sample scenes on a Quest3 Build and then I reproduced it in the Editor as seen above.

Regarding the algorithm, looks to be using hard-coded constants, so not adjustable in settings. But you could possibly alter locally to see if it fixes anything for you: main/org.mixedrealitytoolkit.input/Utilities/HandRay.cs#L73

Thanks for the code reference - I will take a look when I find time. 👍

[AMollis]

@whebertML is going to try using alternative pose source on the far ray interactor on the hands.

[whebertML]

@TheXRMonk Possible fix to this issue with this PR #625

If you would like, you could test the new hand ray pose source to see if it addresses the issue you were seeing.