How many joints are used for testing MPI-INF-3DHP?

[luzzou]

Thanks for your great work! I have question about which joints are used to test the generalization ability on the MPI-INF-3DHP dataset, since it was not mentioned in the original paper.

Looking forward for your reply!

[chaneyddtt]

Hi @trumDog, The full joint list for the MPI-INF-3DHP dataset is {'spine3', 'spine4', 'spine2', 'spine', 'pelvis',
'neck', 'head', 'head_top', 'left_clavicle', 'left_shoulder', 'left_elbow', 'left_wrist', 'left_hand', 'right_clavicle', 'right_shoulder', 'right_elbow', 'right_wrist', 'right_hand', 'left_hip', 'left_knee', 'left_ankle', 'left_foot', 'left_toe', 'right_hip' , 'right_knee', 'right_ankle', 'right_foot', 'right_toe'}. And I used index [25, 24, 23, 18, 19, 20, 4, 3, 5, 7, 16, 15, 14, 9, 10, 11] to keep consistent with the joint list of MPII.

[luzzou]

25, 24, 23, 18, 19, 20, 4, 3, 5, 7, 16, 15, 14, 9, 10, 11

Thank you for your kindly reply! In fact, I have tried to test the generalization ability of the pretrained MDN model on MPI-INF-3DHP using the joints as you mentioned above, but I cannot get reasonable results. The accuracy is low. The details are as follows:

Firstly, I directly use the normalized 2D GT keypoints from the MPI-INF-3DHP validation set as the model's input, and the normalization method is consistent with H36M. The joint indexes are mapped to be consistent with H36M, too. Then, the predicted multiple 3D poses are unnormalized according to the mean and standard derivation computed by the ‘univ_annot3' term from MPI-INF-3DHP validation set, and the joint indexes are mapped back to MPI-INF-3DHP. Finally, I compute the 3DPCK between the results from step 2 and ’univ_annot3'.

I don't know what's the problem, and this question has bothered me for a long time, and I have not found an answer elsewhere. Can you give me some help to correct this? or tell me your testing method?

Really need your help Thanks

[chaneyddtt]

Your description sounds ok, have you ever visualized the estimated poses and are they looks reasonable. Btw, the numbers reported in our paper is based best hypothesis.

[luzzou]

Your description sounds ok, have you ever visualized the estimated poses and are they looks reasonable. Btw, the numbers reported in our paper is based best hypothesis.

@chaneyddtt Hi, I have tested all the five hypotheses, and choose the best results. However, I can only get the best 3DPCK of 54.41, and the predicted 3D poses look very different from the ground truth.

e.g.,

this is the model output

this is the corresponding GT

the results of the Z-dim is the worst and I don't know how to solve this problem

[chaneyddtt]

Seems that you have a different joint index (with the 14th joint being the root joint)?

[luzzou]

Seems that you have a different joint index (with the 14th joint being the root joint)?

Yes, you're right. I use joint indexes as:

Left: MPI-INF-3DHP Right: Human3.6M

and the "head" joint corresponding to "neck" in Human3.6M is ignored.

The test set was downloaded from the official website

[luzzou]

Hi~ sorry to disturb you again, can you share your code to testing the MPI-INF-3DHP dataset?

[chaneyddtt]

Sorry that I do not have time to clean up the code for this part currently. Can you try to use the statistics of 'annot3' instead of 'univ_annot3' to unnormalize the output?

[luzzou]

Sorry that I do not have time to clean up the code for this part currently. Can you try to use the statistics of 'annot3' instead of 'univ_annot3' to unnormalize the output?

I have tried this, but the results are still far from the ground truth, are there any other tricks?

[chaneyddtt]

I did not use other tricks here. I use the mean and std of the 'annot3' data for normalization and unnormalization, and then use a threshold of 150mm (which corresponds to roughly half of head size) to compute the PCK.

[luzzou]

I did not use other tricks here. I use the mean and std of the 'annot3' data for normalization and unnormalization, and then use a threshold of 150mm (which corresponds to roughly half of head size) to compute the PCK.

Did you only use the valid set when calculating statistics? or the training set and the valid set are used together?

[chaneyddtt]

Hi @trumDog, Thanks for pointing out, I used the statistics of the valid set for the normalization. I tried the statistics of human3.6 just now, and it gives a lower pck. I think there is domain shift between those two datasets, and it would make more sense to use the protocol 2 to show the results if you want to test the generalization capacity of your network.

[luzzou]

k there is domain shift between those two datasets, and it would make more sense to use the protocol 2 to show the results if you want to test the generalization capacity of your network.

Thank you very much! I would try it again.