How to get 3D hand joints position coordinate under the camera system?

[buaacyw]

Thanks for your great work! And I have a question: I have the camera intrinsic of the input image and want to get hand joints position coordinate under the camera system

I have read joint order in your doc and the code in hand_mocap_api.py.

I find that your regressor outputs the hand joints position under SMPL 3D coordinate and then you convert them in the order of SMPL 3D coordinate <-> 2D bounding box coordinate<-> original 2D image coordinate. And the Z axes of these different representations are all relative values. As can be seen below, the 5(Index_00) is the origin point.

So I don't know how to get the 3D hand joints position coordinate under the camera system since I don't know the Z axes value of the joint Index_00.

Please help me, thanks for your work again!

[penincillin]

@buaacyw That's a good question. Our models (both body and hand) only predicts relative 3D coordinates, with root set to zero. To obtain absolute coordinates under camera space, I recommend RootNet.

[buaacyw]

@penincillin Thanks for your reply. But RootNet can't be applied on hands.

[Arpitrf]

@buaacyw Have you tried RootNet with hands? Was wondering why it can't be applied to hands?