search

facebookresearch / frankmocap

A Strong and Easy-to-use Single View 3D Hand+Body Pose Estimator
Other
2.12k stars 373 forks source link

About hand pose #136

Closed YuyanHuang closed 3 years ago

YuyanHuang commented 3 years ago
  1. Is pred_hand_pose the local axis-angle? Contains wrist and fingers.
  2. Is the pose of the dataset FreiHand and HO3D the local axis-angle? Contains wrist and fingers.
penincillin commented 3 years ago
  1. pred_hand_pose is represented in relative/local axis-angle.
  2. Annotations of FreiHand and HO3D are also axis-angle. One thing to note is that the input pose parameters of SMPL-X (MANO) model is not exact the joint angles but the joint angles minus mean joint angles. I refer to check origin code for details. Based on this, FreiHand annotations are joint_angles - mean while HO3D annotations are origin joint_angles.
penincillin commented 3 years ago

Question-2 is a little bit tricky. I would suggest you to visualize the FreiHand annotations and HO3D annotations, using the SMPL-X's official implementation instead of the implementation provided by these two datasets. Please feel free to raise questions here if you encounter with any more problems.

YuyanHuang commented 3 years ago

When supervising the axis-angle, have you supervised the wrist joints?

penincillin commented 3 years ago

@YuyanHuang Following the practice of previous works (HMR etc), we do not supervise wrist joints in either 3D joint loss or axis-angle loss. The reason for not supervising 3D joint is that we only predicts relative 3D joints and wrist is used as the root. The reason for not supervising axis-angle is that rotation of wrist is the rotation of overall hands. However, overall hand rotation is correlated to camera intrinsic, which is agnostic in our setting. Therefore, we do not apply axis-angle supervision on wrist rotations. Besides, we have done experiments to incorporate wrist incorporating during training, the performances are not improved.