duanzhiihao / RAPiD

RAPiD: Rotation-Aware People Detection in Overhead Fisheye Images (CVPR 2020 Workshops)
http://vip.bu.edu/rapid/
Other
213 stars 63 forks source link

Question Regarding AP score #21

Closed GivralNguyen closed 3 years ago

GivralNguyen commented 3 years ago

In your paper , i can see that you achieve , 0.919 Precision and 0.951 Recall with confidence 0.3 . However, you achieve AP50 of 96.7. How is this possible when AP is the area under the precision-recall curve ? . Let's assume that for all value of confidence > 0.3, you achieve precision = 1. And for all value of confidence <= 0.3, you achieve precision = 0.919. Then your AP will only be 0.9193 + 0.10.7 = 97.3, only slightly higher than the published 96.7 AP in your paper. However, with huge drop-off in precision when confidence <0.3, i have problems understanding the published AP. Please correct me if i'm wrong

duanzhiihao commented 3 years ago

I'm not sure I understand your question, so let me restate how do I compute AP (I'm following the COCO convention).

Given that we achieve 0.919 Precision and 0.951 Recall when conf=0.3, our AP will be >= 0.919*0.951=0.873. Together with additional area when conf > 0.3 (upper left red part) and conf < 0.3 (bottom right red part), the final AP is 96.7.

Note that, the P-R curve is not continuous since in the COCO convention, they sample 101 Recall values and use the averaged Precision corresponding to these 101 Recalls to approximate the area under the P-R curve.

image

GivralNguyen commented 3 years ago

I see. My bad for not reading clearly about COCO AP convention. Thanks for your detailed answer

duanzhiihao commented 3 years ago

No problem, I’m glad that it’s clear now. thank you for your interest!