search

facebookresearch / ContactPose

Large dataset of hand-object contact, hand- and object-pose, and 2.9 M RGB-D grasp images.
http://contactpose.cc.gatech.edu/
MIT License
338 stars 33 forks source link

About obtaining the key point coordinates of the hand in the camera coordinate system #8

Closed YuyanHuang closed 3 years ago

YuyanHuang commented 3 years ago

First, thank you for your great work! The following is the calculation of the coordinates of the key points of the hand in the camera coordinate system. Is it correct? Do other transformations need to be done? For example, affine transformation. # X: Nx3 P = _cTo[camera_name][frame_idx][:3] X = np.vstack((X.T, np.ones(len(X)))) x = P @ X x = x.T

samarth-robo commented 3 years ago

Hi @YuyanHuang your equations correctly transform X from the object coordinate system to the camera coordinate system.

The affine transformation is only involved when you want to further transform the 3D points in the camera coordinate system to image pixel locations. The chain of transformations and the variable names is mentioned here: https://github.com/facebookresearch/ContactPose/blob/master/docs/doc.md#transform-tree.

A = ContactPose.get_A(camera_name) gives you the affine matrix. You should use A while going from 3D to 2D, and np.linalg.inv(A)while going from 2D to 3D.

I hope that answers your question, but please let me know if not!

YuyanHuang commented 3 years ago

@samarth-robo Yes, that's the answer I want. Thank you for your kind reply. I will close this issue.

YuyanHuang commented 3 years ago

After converting the object coordinates to the camera coordinate system, is the unit in meters?

samarth-robo commented 3 years ago

_cTo is in meters. So if your input X is in meters, the answer is yes. If your X is ContactPose.hand_joints(frame_idx), that is in meters, yes.

YuyanHuang commented 3 years ago

Yes, I got it. Thank you.