Detector recall and PR Curve

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )

http://pjreddie.com/darknet/

Other

21.73k stars 7.96k forks source link

Detector recall and PR Curve #1864

Open gnoya opened 5 years ago

gnoya commented 5 years ago

Hi, I'm testing my detector and I want to get the precision-recall curve (PR Curve) and I had some questions:

Does darknet offer a way to calculate it? If not, how do you recommend doing it?
I saw the command detector recall and used it, but I'm not sure if I'm understanding the results:

3070 7356 7715 RPs/Img: 3.51 IOU: 77.85% Recall:95.35%

Does that mean that from 7715 objects in the dataset, I got 7356 with more than 50% IoU?. Is there anyway to check the precision of those detections using this function (so I could get the PR curve)?

Thank you!

AlexeyAB commented 5 years ago

@gnoya Hi,

un-comment this line: https://github.com/AlexeyAB/darknet/blob/2c5e383c04655fe45f3f533eb3a69a80acbf3561/src/detector.c#L860
recompile Darknet
run this command: ./darknet detector map data/obj.data yolo-obj.cfg backup\yolo-obj_9000.weights

At the end you will see 11 values of Precision and Recall for each class. So you can build PR-curve for each class by using these 11 points.

Don't use command detector recall

hahakid commented 5 years ago

@AlexeyAB I randomly choose this issue to report a likely bug. when using detector map, I can get a mAP, and APs on each class. when using detector map -points 11, I will get a lower mAP, and lower APs on each class. when using detector map -points 21, the mAP and APs increase slightly. what's weird. Please try on your local machine with any cfg and weights file.