aliborji / DetectionUpperbound

Code for calculating the upper bound AP in object detection
81 stars 4 forks source link

Some questions about paper #2

Open xytjcxy opened 4 years ago

xytjcxy commented 4 years ago

Hello,my researchs may need to use some of the concepts in your paper. Therefore, here I have some questions need your answer.

  1. How do you determine the upper bound of object detection? In the paper, you said, the classification score is considered as the detection score. Therefore, does it mean that the best performance of classifier is the upper bound of the detector?

  2. In the section of Error Diagnosis,you quantify the contribution of errors in object detection. However, can you provide the codes about clear the errors, such as removing, correcting or adding detections when appropriate.

1577272412(1)

Looking forward to your response!

Sincerely

aliborji commented 4 years ago

How do you determine the upper bound of object detection? In the paper, you said, the classification score is considered as the detection score. Therefore, does it mean that the best performance of classifier is the upper bound of the detector?

Well, not exactly. Technically, because in detection score (mAP) confidence matters whereas in classification it does not. Read how AP is calculated!

  1. Added a jupyter notebook. The graph u pasted is calculated using the COCO API. Our code generates numbers shown in tables in the paper.
xytjcxy commented 4 years ago
  1. In object detection, the metric to determine positive/negative is whether the IoU >= threshold u. In classification, the metric is whether the confidence >= threshold u.

Therefore, my problem is that whether the AP of classification is the upper bound of AP of the detector.

  1. Thanks for your codes, I think I have figured out how you did it!