GodotVR / godot_openxr

(Maintenance mode) OpenXR drivers for the Godot Game Engine v3.x
MIT License
278 stars 45 forks source link

Feature request: lower level OpenXR-centric hand tracking API #206

Open goatchurchprime opened 2 years ago

goatchurchprime commented 2 years ago

There are two Godot interfaces to the hand-tracking features, OpenXRSkeleton and OpenXRHand, both of which set a series of relative transforms to the components of the tree of finger bones entities, and return no further data. See for example https://github.com/GodotVR/godot_openxr/blob/0d2ea910fdb761ea9c28eb51e78e6e6ebc0fa210/src/gdclasses/OpenXRHand.cpp#L133

However, the the OpenXR HandJoint spec provides absolute transforms for each finger joint relative to the ARVROrigin, as well as joint radii (distance from the joint position to the surface of the skin of the palm if you multiply by the -y vector), and cartesian and angular velocities.

It might be easier to develop working code and the hand skeletons if we accessed the transforms provided by the OpenXR interface directly without this processing and multiplying out, because it might not be quite right. For example, the original quest plugin returned a list of quaternion orientations for the bones (instead of full transforms), which meant that the hand was never distorted by any bones being scaled/stretched. The orientations should always point back to the previous joint, but we could use the distance to select different hand meshes (eg child's hands).