search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.65k stars 7.96k forks source link

Understanding mAP results #2721

Open HamzahNizami opened 5 years ago

HamzahNizami commented 5 years ago

Hi, I have recently trained the YOLOv3 tiny model on my dataset and got what i believe to be fairly good results. I just wanted help in understanding and making sure that is so.

After running the weights using the command "map" at a IoU threshold of 50% I have gotten a mAP value of 73.14%. I am able to increase value by reducing the IoU threshold to 10% to give me a value of 85.29%. I was wondering if these mAP values are in fact good.

I also tried all the other IoU thresholds between 0-100 with increments of 10% each time. These gave me and an average of 49.28%. Below are the results at each 10% increment.

10% = 85.29% 20% = 85.11 30% = 84.91% 40% = 82.13% 50% = 72.14 60% = 53.50% 70% = 23.91% 80% = 4.27% 90% = 0.51% 100% = 0%

There is a fairly dramatic drop between 50% - 100%, however, when reading around, it may be due to the fact my images require a fairly precise location of objects and so does better at a lower threshold.

I am currently planning on trying out the YOLOv3 main model and then spp model(?). Any information and guidance regarding those would be awesome!

Finally, when looking at the official site for YOLO I was confused by the following passage.

Comparison to Other Detectors YOLOv3 is extremely fast and accurate. In mAP measured at .5, IOU YOLOv3 is on par with Focal Loss but about 4x faster. Moreover, you can easily tradeoff between speed and accuracy simply by changing the size of the model, no retraining required!

I was wondering how exactly to achieve greater accuracy, and what is meant by "changing the size of the model, no retraining required!". Does this mean I should be able to use my "YOLOv3-tiny" weights on the other YOLO models - YOLOv3 and YOLOv3.spp directly, without having to train those models on my dataset again?

Thanks in advance, Any advice would be highly appreciated.

@AlexeyAB

AlexeyAB commented 5 years ago

@HamzahNizami Hi,

I was wondering how exactly to achieve greater accuracy, and what is meant by "changing the size of the model, no retraining required!".

It means, that if you trained yolov3-tiny.cfg with width=416 height=416 random=1, then after training you can set width=608 height=608 in yolov3-tiny.cfg and use the same yolov3-tiny.weights file to increase accuracy (mAP).

If your objects are small, then try to train yolov3-tiny-3l.cfg with width=832 height=832 random=1 (or even 1024x1024) Otherwise try to train yolov3-spp.cfg model.