Reproduced results using the middle root joint

[gaogehan]

Thanks for your great work. I try to calculate the results using the middle root joint. MRRPE and MPVPE are correct, but the value of MPJPE increased (MPJPE for all hand sequences: 18.55 mm). I only make changes to the following code when calculating MPJPE.

1. add the following code after https://github.com/facebookresearch/InterWild/blob/main/data/InterHand26M/InterHand26M.py#L39 self.joint_set['middle_root_joint_idx'] = {'right': self.joint_set['joints_name'].index('R_Middle_1'), 'left': self.joint_set['joints_name'].index('L_Middle_1')}

2. add the following code after https://github.com/facebookresearch/InterWild/blob/main/data/InterHand26M/InterHand26M.py#L397

   joint_gt[self.joint_set['joint_type'][h],:] = joint_gt[self.joint_set['joint_type'][h],:] - joint_gt[self.joint_set['middle_root_joint_idx'][h],None,:]
   joint_out[self.joint_set['joint_type'][h],:] = joint_out[self.joint_set['joint_type'][h],:] - joint_out[self.joint_set['middle_root_joint_idx'][h],None,:]

[mks0601]

Hi, MPJPE should be calculated with setting the root joint as the wrist. Setting Middle_1 as the root joint could decrease the MPJPE with the same 3D pose as it make the longest length of the kinematic chain shorter.

[gaogehan]

Thank you for your reply. MPJPE for all hand sequences: 13.09 mm (using the root joint as the wrist). MPJPE for all hand sequences: 18.55 mm ( setting Middle_1 as the root joint using the above code). I am very confused that no decrease in error was observed in MPJPE as seen in the reproduced MPVPE in the paper when using the middle root joint. As many papers use the middle root joint to calculate both MPVPE and MPJPE, I would like to know the metric results for Interwild using this setting. When I reproduce the MPVPE results using the middle root and scale alignment, I observe error decrease as described in the paper. However, the same way to calculate MPJPE does not work.

[mks0601]

Hi, sorry I'm not sure I clearly got you. no decrease in error was observed in MPJPE as seen in the reproduced MPVPE in the paper when using the middle root joint. -> the table in the paper is about IntagHand's reproduce, not about InterWild. You're currently testing with InterWild.

[gaogehan]

Yes, i am currently testing with InterWild. Interwild should exhibit a similar trend when evaluated using the middle root. When I use the middle root to evaluate Interwild, the MPVPE metric decreases, but the MPJPE does not.

[mks0601]

Got it. One thing is that you need to cancel some annotations where middle_root_joint is not valid. Currently, the dataloader cancel annotations where wrist is not valid.

[gaogehan]

Thank you for your prompt reply. I have tried cancel the invalid middle root joint annotations in the dataloader:

            joint_trunc[self.joint_set['joint_type']['right']] *= joint_trunc[self.joint_set['middle_root_joint_idx']['right']]
            joint_trunc[self.joint_set['joint_type']['left']] *= joint_trunc[self.joint_set['middle_root_joint_idx']['left']]

            joint_valid[self.joint_set['joint_type']['right']] *= joint_valid[self.joint_set['middle_root_joint_idx']['right']]
            joint_valid[self.joint_set['joint_type']['left']] *= joint_valid[self.joint_set['middle_root_joint_idx']['left']]

The results for MPJPE is slightly decrease(MPJPE for all hand sequences: 17.32 mm). However, it does not achieve the expected results ( MPJPE less than 13.09 mm (testing with wrist)).

[mks0601]

That is weird. I think what you did is correct. By the way, MPVPE decreases when changing the root joint from the wrist to the middle one?

[gaogehan]

Yes，MPVPE decreases from 10.9 to 7.8.

[gaogehan]

When you were reproducing IntagHand using Middle Root, did the MPJPE decrease?

[mks0601]

I haven't tested with MPJPE.

[gaogehan]

Follow-up: When calculating MPJPE, I replaced the original joint_gt = annot['joint_cam'] with gt of mano_joint_cam, and the metric became normal. Thank you. So I will close this issue now.

[mks0601]

Oh I think the joint set is different. Great thanks!