Request for Metric Calculation Code

[Learner209]

Hi there! 👋

First of all, thank you for open-sourcing EgoAllo and providing such detailed documentation. Your work on estimating body and hand motion in an ego-sensed world is truly inspiring and impressive!

I've been going through the repository and noticed that the code for calculating the quantitative metrics mentioned in the paper (such as MPJPE, PA-MPJPE, GND, and T-head) isn't currently included. These metrics are crucial for understanding and reproducing the performance results reported in your work.

I was wondering if you'd be willing to open-source this part of the code as well? Having access to the metric calculation scripts would be incredibly helpful for:

Reproducing the results reported in the paper
Gaining a deeper understanding of how the performance is evaluated
Enabling fair comparisons with other methods in the field

I understand that research code can sometimes be complex and may require additional cleanup before sharing. If that's the case, even a simplified version or pseudocode for the metric calculations would be greatly appreciated.

Thank you for considering this request. Your work is already contributing significantly to the field, and sharing these additional details would further enhance its impact and reproducibility.

Looking forward to your thoughts on this!

Best regards, Learner209 !

[brentyi]

Hi @Learner209, sorry for the late reply here!

Yeah, unfortunately the eval infrastructure lives in an older and crazier version of the codebase, so it will take nontrivial effort to release in a runnable state. If you want to check just implementation specifics, however, I'm happy to share the eval scripts in their (unrunnable) state: https://gist.github.com/brentyi/188e1468aa56203fac14ed3e24df2160

• The metrics themselves are mostly straightforward: MPJPE, after all, is just np.mean(np.linalg.norm(ground_truth_joints - estimated_joints, axis=-1)).
• Some of the metrics we computed, like foot_skate and head_ori, were computed in the code but skipped in the paper (mostly for space).
• Some metrics may have been renamed: GND in the paper, for example, is called foot_contact in the code.

We do plan to make some updates to the released version of the code to include things like WiLoR integration; I can do my best to include some metrics code when we do that!

[Learner209]

Hi! Thanks for your response and the details. I appreciate the offer to share the eval scripts, even the implementation specifics can help a lot! This project is really awesome.