search

mks0601 / Hand4Whole_RELEASE

Official PyTorch implementation of "Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation", CVPRW 2022 (Oral.)
MIT License
315 stars 31 forks source link

Hand pose weird results #44

Open sylyt62 opened 2 years ago

sylyt62 commented 2 years ago

I'm running a real-time application to detect hands and drive a standard smplx model (default flat hand pose) to act, but got some weird stretch for both hands. Like, all fingers move to the wrong side. Any idea?

I set flat_hand_mean=True here, https://github.com/mks0601/Hand4Whole_RELEASE/blob/21ce3c03a4018d6b97324e0df061f026e4d276c0/common/utils/human_models.py#L80

and tried mano.layer['left'] here, https://github.com/mks0601/Hand4Whole_RELEASE/blob/21ce3c03a4018d6b97324e0df061f026e4d276c0/main/model.py#L29

1 2

mks0601 commented 2 years ago

You should set flat_hand_mean=False

sylyt62 commented 2 years ago

Actually nothing changes when changing this parameter. The rotation keeps the same

mks0601 commented 2 years ago

That option adds some constant rotation to the input hand joint rotations in SMPLX layer. Although the estimated rotation is the same, SMPLX applies different rotations by adding a constant rotation.

sylyt62 commented 2 years ago

I mean, after using the default flat_hand_mean=False, the displayed pose keeps almost the same as above pictures.

mks0601 commented 2 years ago

Also, you should use right hand mano model. Hand4Whold is trained to output right hand 3D pose. The left hand images are flipped to the right hand images.