Question about benchmark AP and AR

jin-s13 / COCO-WholeBody

ECCV2020 paper "Whole-Body Human Pose Estimation in the Wild"

767 stars 73 forks source link

Question about benchmark AP and AR #18

Closed jacopogiuliani closed 3 years ago

jacopogiuliani commented 3 years ago

Good evening. First of all, many thanks and compliments for your work. I have a few questions about the AR and AP values given in the COCO-WholeBody Benchmark table:

Is AP as intended for standard COCO dataset evaluation? That is, the mean of AP for each class over 10 IoU threshold values from 0.5 to 0.95 with 0.05 as the step?
Is AR computed with 100 maximum predicted boxes for each image and no area restriction on these boxes?

thanks in advance and have a good day.

luminxu commented 3 years ago

Thanks for your interest. We follow the COCO keypoint dataset for evaluation, which uses 10 thresholds for mAP and 20 detections for AR. Please refer to kpts_eval@coco for more details.