Left Hand Landmark for MP pose and MP hand

[UtsaChattopadhyay]

I am using both mediapipe pose and mediapipe hand models to get landmarks and retarget to an AVATAR for animation. But the LeftHand (15) and RightHand (16) landmark from mediapipe POSE and wrist landmark (0) from mediapipe Hands is not the same for one image, and so while creating the BVH, for the whole body including hand bones, I am getting wrong rotation for the hand part. Both are working fine seperately. May I know is there any way to map the HandLandmark origin to the pose Landmark one? Thanks

[kuaashish]

Hi @UtsaChattopadhyay, Could you please elaborate your query with complete details and platform which you are referring to. Thank you!

[UtsaChattopadhyay]

Yeah sure. The link below explains a bit on this. https://github.com/google/mediapipe/issues/3491#issuecomment-1185161389 So I am using mediapipe pose and mediapipe hand together to generate landmark and then later create Euler rotations to make BVH out of it. But the LeftHand landmark of mediapipe pose here is calculated with hips at the center and for the hand at the HANDS GEOMETRIC CENTER. So i want to map the hand calculated WRIST landmark to the one calculated already by the POSE model to merge both together. If you have any isight on this please let me know. Thanks.

[UtsaChattopadhyay]

If you refer to the picture, thats how it looks if I use the Landmark value for LeftHand from two models. I just want to map the Hand model landmark output to Pose model. Any help is appreciated.

[UtsaChattopadhyay]

Hi, I have solved the issue, so tentatively you have to do this MediaPipeHolistic:BODY_CENTERED_HAND_COORDINATES = MediaPipePose:POSE_WORLD_LANDMARKS:AverageValueOfHandCoordinates + MediaPipeHands:MULTI_HAND_WORLD_LANDMARKS Thanks.

[kuaashish]

Hi @UtsaChattopadhyay, Glad to know the issue has been resolved! Could we move this issue to the close status as resolved. Thank you!

[UtsaChattopadhyay]

Yeah sure

On Tue, Nov 29, 2022, 14:46 kuaashish @.***> wrote:

Hi @UtsaChattopadhyay https://github.com/UtsaChattopadhyay, Glad to know the issue has been resolved! Could we move this issue to the close status as resolved. Thank you!

— Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/3879#issuecomment-1330158562, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKS5W3IQI5LFNFBFN3I2QWLWKWRDBANCNFSM6AAAAAASHJ3KCE . You are receiving this because you were mentioned.Message ID: @.***>

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No