Open talpadk opened 3 months ago
If I apply the following Euler rotations to the Armature nodes: Left hand: [0, 90, -90] Right hand: [90,90,0]
The hands are oriented correctly inside VR. (The fingers still bends the wrong way)
Which gives the rotation matrices Left: [ 0.0000000, 0.0000000, 1.0000000; -1.0000000, 0.0000000, 0.0000000; -0.0000000, -1.0000000, 0.0000000 ]
Right: [ 0.0000000, 0.0000000, 1.0000000; 1.0000000, 0.0000000, -0.0000000; -0.0000000, 1.0000000, 0.0000000 ]
So it seems like the coordinates from the Pico 4 should be rearranged slightly
And for the left hand two of them used as negative values
If I use bone update full on the left hand that uses the depreciated OpenXRHand it renders in the correct orientation without any rotation of the armature.
But then it distorts the hand mesh
Thanks for the report!
Is this a regression (meaning, it works in Godot 4.2, but is broken in 4.3-beta1)?
If there is a bug here, it's likely in Godot itself, rather than this extension. The extension is really only responsible for the loader and Android manifest changes - basic OpenXR hand tracking support is in Godot.
I will move the bug report to the Godot tracker instead.
Should the part about the Pico not picking up on the application supporting hand tracking remain here or is that also a Godot issue?
OpenXRHand behaves the same in Godot 4.2.2 (I can't select to only use rotational transform but other than that) XRNode3D + XRHandModifier3D isn't available as far as I can tell in 4.2.2
And thanks for your help this far
I will move the bug report to the Godot tracker instead.
I think it's OK to keep it here until we figure out what's going on. If this is a matter of adding something to the Android manifest or requesting a Pico-specific permission, then the fix will be here. But if it's anything else, then the fix will need to be in Godot.
Try and see if you can reproduce the issue in this demo: https://github.com/godotengine/godot-demo-projects/tree/master/xr/openxr_hand_tracking_demo
For me when I tried that on my PICO 4 it was working correctly.
Do note that the PICO 4 only provides hand tracking data if you are not using a controller, it does not support controller inferred hand tracking.
Also seeing the mention of 3.0.0 beta 1 being used, I am not sure if we added the PICO changes in that yet. We added additional manifest entries in beta 2 and moved to the Khronos loader in beta 3 adding additional required manifest entries.
Got stuck way too long on vendors beta 3 requires godot beta 3, drained all the fun out of it, will have to try reporting back later.
I believe that the hand tracking on pico 4 has a bug or two, or possibly I'm using OpenXR the wrong way I'm using:
I have tied both the depreciated OpenXRHand and XRNode3D + XRHandModifier3D
In the export settings I have tired bot optional and required for hand tracking
In the project settings i have enabled hand tracking and tried both with and without "Hand Interaction Profile" enabled
Some of the rotation and bending issues might be solvable by applying a rotation matrix to the data from the Pico 4 or perhaps even just swapping an axis or two. It may of cause simply be a bug in the Pico firmware, but it would be nice if it could be worked around until they fix it (if ever).
As to why the XRHandModifier3D doesn't modify the bones I wouldn't rule out a misconfiguration on my part
Oh and last but not least: Thanks for the plugin quite amazing how relatively easy it is to code VR applications using it