mordentral / VRExpansionPlugin

A UE4-5 VR framework
https://www.vreue4.com
MIT License
543 stars 113 forks source link

GetOpenXRHandPose only returns empty hand joing transforms due to empty GripRotation/GripPosition #22

Closed Mystfit closed 2 years ago

Mystfit commented 2 years ago

https://github.com/mordentral/VRExpansionPlugin/blob/9b27c6320cad22d32182b71bbaa36f18f58930c2/OpenXRExpansionPlugin/Source/OpenXRExpansionPlugin/Public/OpenXRExpansionFunctionLibrary.h#L83

UE5.0 from the store and running on the Quest 2.

The hand key transforms are valid, but the GripRotation and GripPosition values used to construct ParentTrans are set to 0, and so the result of GetRelativeTransform(ParentTrans) is also 0 which results in the skeletal transforms not being saved correctly.

Mystfit commented 2 years ago

Upon digging further into FOpenXRHandTracking::GetControllerOrientationAndPosition(), it uses either Left or Right as MotionSource arguments instead of LeftGrip and RightGrip specified in FOpenXRHMD::GetMotionControllerData() which is why I think it is returning empty postion and rotation values. I'm going to try substituting the GripSources and seeing what happens.

Update: I replaced the ParentTrans variable with the following ternary to swap between either the grip pose for a controller or the palm handkeypoint if using hand tracking and my fingers are now moving on both hands mapped to a full body Mannequin. FTransform ParentTrans = MotionControllerData.DeviceVisualType == EXRVisualType::Controller ? FTransform(MotionControllerData.GripRotation, MotionControllerData.GripPosition, FVector(1.f)) : FTransform(MotionControllerData.HandKeyRotations[(uint8)EHandKeypoint::Palm], MotionControllerData.HandKeyPositions[(uint8)EHandKeypoint::Palm], FVector(1.f));

mordentral commented 2 years ago

Well thats quite the annoyance that they don't fill out the tracking variables there, i'll look at it tonight

mordentral commented 2 years ago

Ported to 4.27 + with some changes to make it a bit more clear, thank you for the report.

https://github.com/mordentral/VRExpansionPlugin/commit/d666608b639162a664b486a64457b07796d36b92