why my mAP is close to 1？

[gooodooo]

I use YOLOv3 to training on my own dataset which only contains 3 kinds of objects.And I delete the 'difficult' information which I don't if it is the reason to cause this warning.Because npos = 0 and I delete "npos = npos + sum(~difficult)" in the end : rec = tp / float(npos) So I think the missing of "difficult" is the reason to cause the runtime warning. But my dataset is KITTI which dosen't contain the "difficult" information. So how can I avoid this error since I think the mAP is wrong.

[gooodooo]

I modify the line ‘npos = npos + sum(~difficult)’ to ‘npos = npos + len(R)’,then the code runs well,but I still don't if this code is right..Because my mAP is 0.611 which is a little lower than my expectation....

[AlexeyAB]

mAP = 0.98 is very well, it means that mAP = 98%.

• Did you train Yolo v3 on KITTI dataset?
• Do you try to get mAP for validation part of KITTI dataset?
• What mAP can you get by using command like this? ./darknet detector map data/obj.data yolo-obj.cfg backup\yolo-obj_7000.weights in the obj.data use valid=valid.txt or valid=train.txt

[gooodooo]

Thank you very much ! I trained Yolo v3 on KITTI dataset, and I try to gat mAP for KITTI. I ran ./darknet detector map data/obj.data yolo-obj.cfg backup\yolo-obj_7000.weights in the obj.data use valid=valid.txt ,mAP = 60.27 in the obj.data use valid=train.txt ,mAP = 63.94 But I think this mAP is a little low,because when I train the model ,IOU is very high I still doubt the code modified is wrong:‘npos = npos + len(R)’。。。

[AlexeyAB]

in the obj.data use valid=valid.txt ,mAP = 60.27

So your currently mAP = 60.27%

IoU during training (detector train) and during detector recall isn't correct, more precisely, this is a very specific IoU, that is only needed for internal use by the developer of the Darknet framework.

How many iterations did you train? Did you change learaning_rate? And what radnom, width and height did you use?

[gooodooo]

1. I trained 56000 iterations with the default anchors, after that I changed the anchor and trained another 42000 iterations.

2. I didn't change the learning_rate,because I thought that these lines in yolov3_voc.cfg will change the learning_rate.

   • learning_rate=0.001
   • burn_in=1000
   • max_batches = 120000
   • policy=steps
   • steps=40000,45000
   • scales=.1,.1

3. I set

• random = 0,
• width and height are both 416.

But the pictures in my dataset are 1242 X 375. So how can I improve my training? Thanks A LOT

[AlexeyAB]

1. Set width=1248 height=384 in your cfg-file.

2. Update your code from GitHub, I just added fix for usage random=1 for non-square networks. Then set random=1 in each 3 [yolo]-layer in cfg-file.

3. Re-calculate anchors for 1248 x384 ./darknet detector calc_anchors data/obj.data -num_of_clusters 9 -width 1248 -height 384 and set it in each 3 [yolo]-layer in cfg-file.

And train Yolo v3 from the begining.

[AlexeyAB]

But it will work 2.7x times slower. If Out of memory error will occur, then increase subdivison=16, 32 or 64. Or you can try to set width=608 height=384 and re-calculate anchors - it will work ~ with the same speed: ./darknet detector calc_anchors data/obj.data -num_of_clusters 9 -width 608 -height 384

[gooodooo]

wow! After I set width=1248 height=384,and random=1 , the mAP changed to 80.06!! But I don't really figured out that is this just for thresh = 0.25?Because I thought I have used 11 point mAP. And should I continue to train this model or train from the begining?

[AlexeyAB]

• mAP and AP are for 11 points
• thresh=0.25 is only for IoU, F1, TP/FP/FN

---

After I set width=1248 height=384,and random=1 , the mAP changed to 80.06!!

How many iterations did you train? Or did you just set these param and do detector map?

Try to train from the begining using these params - you should get higher mAP.

[gooodooo]

I just set these param and do detector map.So it's amazing. Yesterday I trained from the begining using these params.After 31000 iterations,I got 83.68 mAP on my valid dataset. But when I was training ,I found that because I set the random = 1,so it resizing the picture in a random size like 448 or 480.. So I was confused that why I set width=1248 height=384? Since it doesn't help to keep high resolution。And should I change my anchors according to the hight and width? Thank you for your reply which really help me a lot!

[AlexeyAB]

So I was confused that why I set width=1248 height=384? Since it doesn't help to keep high resolution

https://github.com/AlexeyAB/darknet/issues/808#issuecomment-388754007

Update your code from GitHub

[gooodooo]

So do I need to change the anchors ?Because now the grid in the last feature map is not 13 X 13 anymore,but 39 X 12.And if I need to change,how?

[AlexeyAB]

Use this command: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width=1248 -height=384

And change anchors in each of 3 [yolo] layers.

[gooodooo]

So,if I want to change the backbone of darknet,which files do I need to modify? And if I want to change the number of anchors,how? Thank you very MUCH!

[AlexeyAB]

So,if I want to change the backbone of darknet,which files do I need to modify?

Modify your cfg-file.

And if I want to change the number of anchors,how?

Recalculate anchors and then in each of 3 [yolo]-layers change:

• https://github.com/AlexeyAB/darknet/blob/132251d72325e1005ef6c47f83d6a4e9b9355d12/cfg/yolov3.cfg#L784
• https://github.com/AlexeyAB/darknet/blob/132251d72325e1005ef6c47f83d6a4e9b9355d12/cfg/yolov3.cfg#L782
• https://github.com/AlexeyAB/darknet/blob/132251d72325e1005ef6c47f83d6a4e9b9355d12/cfg/yolov3.cfg#L781

For example you want to use 3 anchors instead of 9, then:

[yolo]
mask=2
anchors = 10,13,  62,45, 373,326
num=3

...

[yolo]
mask=1
anchors = 10,13,  62,45, 373,326
num=3

...

[yolo]
mask=0
anchors = 10,13,  62,45, 373,326
num=3

[marcunzueta]

Hello @AlexeyAB,

Will the number of anchor boxes, increase or decrease performance or accuracy. In my case, I am just trying to detect license plates (which normally have the same aspect ratio, but in different sizes). I am using a modified version of tiny-yolo now, with 6 anchors instead of 9.

Thank you

[AlexeyAB]

@marcunzueta Hi,

Try to calculate 8 or 10 anchors for your dataset with flag -show, if the values will be very different, then it can increase accuracy. In other cases it will not increase accuracy.

[gooodooo]

I change the backbone in yolov3-voc.cfg. I want all anchors to pridict boxs in one layer,so I set

1. filters=72
3. [yolo]
4. mask = 0,1,2,3,4,5,6,7,8
5. anchors = 34,32, 75,47, 45,101, 128,71, 98,189, 195,108, 267,167, 378,189, 365,347
6. classes=3
7. num=9

but my training broke down.The obj scores get close to 0 like this. Besides,your comand to calculate mAP is not useful for this backbone. So I wonder if there must be 3 yolo layers?

[AlexeyAB]

@gooodooo train more than 2000 iterations

[gooodooo]

hi，thank you very much . my training met a difficult problem.I want to improve the performance on detecting dense objects.Like pedestrian or bird and so on. What should I do? : )

[AlexeyAB]

@gooodooo Use yolov3.cfg and increase width=832 height=832 or width=1024 height=1024

[gooodooo]

HI，I read the paper about YOLOv3,but it doesn't mention the mAP performance on PASCAL VOC dataset.Can you share the mAP of darknet53-416 on PASCAL VOC?Because I want to compare my results to the standard result(especially darknet53-416)! Thank you very much!

[ShoufaChen]

@gooodooo @AlexeyAB Hi, is there any result about mAP for YOLOv3 Pascal voc？

Thank you very much!

[gooodooo]

@ShoufaChen I have not found yet

[AlexeyAB]

@ShoufaChen There is no mAP for Pascal VOC on Yolo v3. You should train and check it by yourself.

[ShoufaChen]

@AlexeyAB ok, thank you.