About performance difference between different `PA_MPJPE` calculation method on hands.

[imabackstabber]

Hi, I adopted reconstruction_error function in utils/metric_pampjpe.py to calculate PA_MPJPE metric on hands(which is originally used for body, in the repo)., and I'm pretty sure the result is aligned(i.e. substract the position of Wrist) But the result is inferior to the de-facto standard when calculating PA_MPJPE for hands, which is written in the code given: saving results as pred.json and using eval.py in freihand to calucalate align result(and that will be the final PA_MPJPE, to my knowledge). The result reconstruction_error given is roughly 24~25mm, however, the pred.json method mentioned above says that result is about 6~7mm. The huge difference bothers me. Could you please explain the reason why the two methods given differ so much? Your reply will be highly appreciated.

[FastMETRO]

Hello,

For the evaluation, we used the official evaluation code provided by FreiHAND. If you want to merge the evaluation code with our repository, please check these guidelines.

---

• The official evaluation annotations should be correctly loaded. In freihand.tar, evaluation annotations are filled with dummy values. At the time when METRO released freihand.tar, the official evaluation annotations were not accessible. Please refer to lines 211-215 of tsv_demo_freihand.py in METRO.

• The 3D hand outputs should be correctly aligned. Please refer to compute_similarity_transform() in src/utils/metric_pampjpe.py and align_w_scale() in the official evaluation code.

---

Thanks for your interest in our work!!

[FastMETRO]

Please reopen this issue if you need more help regarding this.

[imabackstabber]

Thanks! That really helps with my problem!

[FastMETRO]

Glad to hear that our guidelines resolved your issue :)