search

lixiny / CPF

[ICCV 2021] CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction
118 stars 17 forks source link

what's the meaning of "adapt"? #9

Open haolyuan opened 2 years ago

haolyuan commented 2 years ago

I notice that there are hand_pose_axisang_adapt_np and hand_pose_axisang_np in your code. Could you please explain what's the difference between them?

kelvin34501 commented 2 years ago

For code here:

hand_pose_axisang_adapt_np = np.asarray(test_sample[CIAdaptQueries.HAND_POSE])
hand_pose_axisang_np = np.asarray(test_sample[CIDumpedQueries.HAND_POSE])

hand_pose_axisang_adapt_np is the pose from the dataset's annotation (name coming from CIAdaptQueries which stands for an adaptor of the hodataset); hand_pose_axisang_np is the pose from the dump results.

haolyuan commented 2 years ago

Can I get hand_joints_adapt_np by using hand_pose_axisang_adapt_np? When I run demo, I found they are not the same.I have take global trans into consideration.

haolyuan commented 2 years ago

One more question! I would like to know what's the meaning of b_axis = (transf[:, 1:, :3, :3].transpose(2, 3) @ b_axis.unsqueeze(-1)).squeeze(-1) in axislayer.py. I know the meaning of b_axis = hand_joints[:, self.joints_mapping] - hand_joints[:, [i + 1 for i in self.joints_mapping]] is the relative jts vector in mano order.

kelvin34501 commented 2 years ago

Can I get hand_joints_adapt_np by using hand_pose_axisang_adapt_np? When I run demo, I found they are not the same.I have take global trans into consideration.

It depends on the dataset. hand_joints_adapt_np and hand_pose_axisang_adapt_np are annotations from dataset. However, different datasets might use different joint locations. You might need to try ManoAdaptor to see whether you can get the annotated joints.

One more question! I would like to know what's the meaning of b_axis = (transf[:, 1:, :3, :3].transpose(2, 3) @ b_axis.unsqueeze(-1)).squeeze(-1) in axislayer.py. I know the meaning of b_axis = hand_joints[:, self.joints_mapping] - hand_joints[:, [i + 1 for i in self.joints_mapping]] is the relative jts vector in mano order.

The mano pose is in its local frame, so it needs to be transformed to get the axis in global frame.

haolyuan commented 2 years ago

I almost understand. But there is still a question. We know that the mano skeleton goes like J'=R0*R1*...*Rp(J-Jp)+Jp', where p mean the closest parent. So I don't understand what's the meaning of RpT*(J'-Jp'), where T means transpose. I think there must be something wrong about my understanding. Thanks a lot!