openxrlab / xrmocap

OpenXRLab Multi-view Motion Capture Toolbox and Benchmark
https://xrmocap.readthedocs.io/
Other
342 stars 42 forks source link

[Refactor] Refactor metric evaluation #79

Closed wqyin closed 1 year ago

wqyin commented 1 year ago
  1. Refactor matric evaluation including MPJPE, PA-MPJPE, PCK, PCP, mAP, and recall
  2. Update sample configs to control the evaluation for various datasets
  3. Add a prediction matcher when the number of predicted people does not align with the number of ground-truth
  4. Add the panoptic_15 convention, which is a subset of panoptic (19 keypoints)
  5. Remove MVPEvaluation and add End2EndEvaluation for learning-based methods
  6. Add sample config for SMPL-X estimation
codecov-commenter commented 1 year ago

Codecov Report

Merging #79 (2342328) into main (ef116a7) will increase coverage by 0.13%. The diff coverage is 36.25%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main      #79      +/-   ##
==========================================
+ Coverage   61.58%   61.71%   +0.13%     
==========================================
  Files         180      182       +2     
  Lines       11461    11681     +220     
==========================================
+ Hits         7058     7209     +151     
- Misses       4403     4472      +69     
Flag Coverage Δ
unittests 61.71% <36.25%> (+0.13%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
xrmocap/core/train/trainer.py 0.00% <ø> (ø)
xrmocap/model/registrant/smplify.py 85.62% <ø> (ø)
xrmocap/model/registrant/smplifyx.py 90.69% <ø> (ø)
xrmocap/core/evaluation/end2end_evaluation.py 17.24% <9.27%> (+17.24%) :arrow_up:
...core/evaluation/metrics/precision_recall_metric.py 12.76% <12.76%> (ø)
xrmocap/core/evaluation/metrics/pcp_metric.py 14.15% <14.15%> (ø)
...nsform/convention/keypoints_convention/__init__.py 73.78% <15.38%> (-8.44%) :arrow_down:
xrmocap/utils/mvp_utils.py 25.43% <20.00%> (+0.43%) :arrow_up:
xrmocap/core/evaluation/base_evaluation.py 46.15% <33.33%> (+46.15%) :arrow_up:
xrmocap/utils/eval_utils.py 68.75% <42.85%> (-21.25%) :arrow_down:
... and 13 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.