search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.75k stars 7.96k forks source link

mAP and IOU fluctuations #3205

Open ArturRog opened 5 years ago

ArturRog commented 5 years ago

I wonder what can cause such fluctuations of IoU values. The mAP is strangely stable as well. My training set is very small (~450 for training, ~120 for test).

Config file: ` batch=64 subdivisions=8 width=608 height=608 channels=3 momentum=0.9 decay=0.0005 angle=5 saturation = 1.5 exposure = 1.5 hue=.2

learning_rate=0.001 burn_in=1000 max_batches = 10200 policy=steps steps=8000,9000 scales=.1,.1 ` I'm detecting only one class. The angle, saturation, exposure and hue are slightly higher than the default values. I've increased the max_batches, just to see if it will yield any benefit.

IoUs: image

mAP:

image

^ values on screens comes from ./darknet detector map command.

AlexeyAB commented 5 years ago

Since mAP@0.5 is calculated for IoU>0.5 then it doesn't matter for mAP@0.5 is IoU=0.6 or 0.7. May be this is the reason.

ArturRog commented 5 years ago

What would you suggest then? Set lower threshold like -iou_thresh 0.25?

AlexeyAB commented 5 years ago

It depends on what do you want.

I suggest to see on mAP instead of IoU.

ArturRog commented 5 years ago

I'd like to have both metrics. Is this possible? With -iou_thresh 0.2 it is more "stable" (less fluctuations).