Increasing mAP

[pbalaji98]

Hello,

I am trying to train my network on some images and only reach a mAP value of around ~ 73%. Here are the details.

• 1 class, (1-15 objects per image,). Images are grayscale, although I didn’t change the number of channels
• Calculated anchors using kmeans and used them in my cfg file
• Training set - ~6500 images
• Validation set - ~650 images (no crossover with training images)
• mAP value of ~74% after about 1700 iterations. The reason I say that the peak is 74% is because the mAP decreased on the next calculation. I believe that this suggested that I was starting to overfit. My average loss at this time is around ~1.8
• Precision seems to be an appropriate number (> 80%). Recall is fairly low (62%), so this suggests that the reason why my mAP isn’t higher is because my network is not fully learning what an object looks like

Here are the images and what the boxing should look like. I didn’t use YOLO_Mark to generate my annotation files (I wrote a script to do it instead). To ensure that my boxes were correct, I drew the boxes and the centers of those boxwa

Below is my cfg file. I apologize for not being to share any images like the cluster image or the loss chart, but with the VM I’m using to train, I can’t use OpenCV. Do you have suggestions on how to improve the mAP?

yolo_stream.txt

[AlexeyAB]

@pbalaji98 Hi,

. I didn’t use YOLO_Mark to generate my annotation files (I wrote a script to do it instead). To ensure that my boxes were correct, I drew the boxes and the centers of those boxwa

• Did you check your labels by using Yolo_mark or training with flag -show_imgs ?
• What are the circles in your image?

but with the VM I’m using to train, I can’t use OpenCV.

• Why? If it lead to error during training - just run training with flag -dont_show

Do you have suggestions on how to improve the mAP?

Use yolov3-spp.cfg instead of yolov3.cfg Use default anchors. Train at least 4000 iterations. And use subdivisions=32 or 16 if you can.

[pbalaji98]

@AlexeyAB Hello.

The circles are what I drew to denote the center of the boxes, just to make sure that I was annotating correctly.

I can't compile with OpenCV=1 due to not have the OpenCV package. The VM doesn't carry it. I can try seeing if I can install it.

Why would yolov3-spp.cfg be better? Could you explain requesting me to use default anchors and training to at least 4000 iterations?

[pbalaji98]

Bump

[AlexeyAB]

@pbalaji98 Hi,

Check your annotations by using Yolo_mark.

yolov3-spp.cfg is the best model currently.

But you can try to use new models:

• https://github.com/AlexeyAB/darknet/files/3253820/yolo_v3_spp_pan_scale.cfg.txt

• yolo_v3_tiny_pan3_aa_ae_mixup_scale_giou.cfg.txt

• https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-494148968

In many cases it's better to use default anchors, because your re-calculated anchors are not suitable for correspond [yolo]-layers: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

recalculate anchors for your dataset for width and height from cfg-file: darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 then set the same 9 anchors in each of 3 [yolo]-layers in your cfg-file. But you should change indexes of anchors masks= for each [yolo]-layer, so that 1st-[yolo]-layer has anchors larger than 60x60, 2nd larger than 30x30, 3rd remaining. Also you should change the filters=(classes + 5)* before each [yolo]-layer. If many of the calculated anchors do not fit under the appropriate layers - then just try using all the default anchors.

[benkelaci]

Hi Alexey,

What about yolov3-tiny anchor recalculation? What are the recommendations for that? (two scale, which sizes etc.). Thanks.

[pbalaji98]

@AlexeyAB. I used Yolo_mark to check a few of annotations and they seem correct. I also trained with the yolov3-spp.cfg file and didn't get better results. After 4000 iterations, the highest mAP score was %70. The mAP began to fluctuate after 1800 iterations.

[pbalaji98]

Do you have any other suggestions? The fluctuation of the mAP score is very drastic (drop from %66 to %30 in consecutive mAP calculations, for example)

[AlexeyAB]

@pbalaji98 Can you show Loss & mAP chart?

[WilburZjh]

Hi @AlexeyAB

In many cases it's better to use default anchors, because your re-calculated anchors are not suitable for correspond [yolo]-layers: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

recalculate anchors for your dataset for width and height from cfg-file: darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 then set the same 9 anchors in each of 3 [yolo]-layers in your cfg-file. But you should change indexes of anchors masks= for each [yolo]-layer, so that 1st-[yolo]-layer has anchors larger than 60x60, 2nd larger than 30x30, 3rd remaining. Also you should change the filters=(classes + 5)* before each [yolo]-layer. If many of the calculated anchors do not fit under the appropriate layers - then just try using all the default anchors.

1. May I know why in many cases its better to use default anchors? Here's my recalculate anchors: anchors = 34, 33, 45, 38, 48, 52, 73, 68, 101, 89, 98,139, 149,109, 184,175, 302,305 it seems that mine only get a small improvement compared with the default value...
2. Is it necessary to use another clustering methods to replace kmeans?
3. If I want to improve the mAP for small objects, any suggestions rather than changing the values in configuration file?

Best