search

geopavlakos / hamer

HaMeR: Reconstructing Hands in 3D with Transformers
https://geopavlakos.github.io/hamer/
MIT License
326 stars 28 forks source link

Use mano param from hamer to SMPLX hand pose #26

Closed zhangyuhong01 closed 6 months ago

zhangyuhong01 commented 6 months ago

Great work! I'd like to know how to convert the MANO parameters obtained from Hamer into the hand pose of SMPL-X. Directly replacing them results in odd effects.

geopavlakos commented 6 months ago

Can you give examples of what you observe? The pose parameters we regress for HaMeR should be directly compatible with the hand pose parameters that SMPL-X expects as input. Is the problem happening with both hands or only with the left hand? If it's the second, you might need to mirror the parameters (left-right flipping).

zhangyuhong01 commented 6 months ago

https://github.com/geopavlakos/hamer/assets/63392592/7a2b2643-994d-462b-9a20-c465f88fe22f Thank you for your reply! It indeed seems to be with problem in the left hand. so how should I flip the mano parameters for the left hand?

geopavlakos commented 6 months ago

Actually, the right hand doesn't look correct either. Could you try again by setting the flag flat_hand_mean=True?

Regarding flipping, you can convert the rotation matrices to axis angle and then for each joint rotation, you will need to multiply the second and the third element by -1 (see also how we do it in our code here).

ireneMsm2020 commented 3 months ago

same error, I multiply the second and the third element by -1 with ['pred_mano_params']['hand_pose'] and copied into SMPLH, but the left hand still with problem. " convert the rotation matrices to axis angle" is indeed? how can i flip the mano parameters without the rotation matrices to axis angle?