search

geopavlakos / hamer

HaMeR: Reconstructing Hands in 3D with Transformers
https://geopavlakos.github.io/hamer/
MIT License
326 stars 28 forks source link

Extracting 3D hand pose wrt. camera #30

Closed Dhruv2012 closed 5 months ago

Dhruv2012 commented 6 months ago

Hi,

I'm facing an issue in extracting 3D coordinates of hand wrt. camera (in metric scale) in the code. I tried a few things including out['pred_keypoints_3d'], out['pred_vertices'], and other variables in the model's output dictionary. However, the resulting 3D coordinates seem to be inaccurate. Would you mind pointing me to the correct way of doing this?

Thanks in advance!

geopavlakos commented 6 months ago

You can get the 3D coordinates of the hand keypoints or hand vertices in the camera frame by adding pred_cam_t_full to the out['pred_keypoints_3d'] or out['pred_vertices'] respectively. If you check the demo code, pred_cam_t_full is computed using an arbitrary focal length value. If you have access to the camera intrinsics, you can use those to get a better estimate of the hand location.

fnuabhimanyu8713 commented 4 months ago

@geopavlakos also you have to multiply the x-coordinate of the left hand by -1?