Same mAP after adding more images for yolov4

[poornimajd]

Great work on yolov4 @AlexeyAB and team! I used yolov4 for training driving dataset with 16k images for training and 8 classes.I got a test mAP of 84.2% after 16000 iterations.To increase the mAP further I added around 2.5k augmented images to the dataset.The data augmentation used was -left-right flipping,hsv color augmentation,and images obtained by cropping out only one class from a image containing other classes. I trained the model for 20000 iterations with the following changes in yolo-obj.cfg

for training for small objects (smaller than 16x16 after the image is resized to 416x416) - set layers = 23 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L895 set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L892 and set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L989

but I got the same mAP without any improvement and the accuracy actually decreased.Can I know what might have gone wrong. Any suggestion is appreciated.

[AlexeyAB]

Did you train it from the begining or did you resume training? What training command did you use? Show chart.png for the both training steps 1 and 2. Are your new images representative for the old separate validation dataset?

[poornimajd]

Thanks for the quick reply. I trained it from the beginning.Training command: ./darknet detector train ./obj.data ./cfg/yolo-obj.cfg yolov4.conv.137 -dont_show Training step 1 chart: I do not have the training step 2 chart,because I did the training over vpn.

Are your new images representative for the old separate validation dataset?

Almost similar.

[AlexeyAB]

What final mAP did you get?

Maybe your dataset has conflicting labels.

To increase the mAP further I added around 2.5k augmented images to the dataset.The data augmentation used was -left-right flipping,hsv color augmentation,

Darknet does such data augmentation automatically.

and images obtained by cropping out only one class from a image containing other classes.

Show examples.

May be your new images don't bring any new information.

[poornimajd]

@AlexeyAB Thank you for the reply. I got a final mAP of 84%.

Maybe your dataset has conflicting labels.

I checked the data with the help of labelimg tool.

Darknet does such data augmentation automatically.

Yes this is true,but I wanted to increase the number of images for minority class and I concentrated on those images where there was more of occlusion.

Show examples.

Here are two images: This is an image of a truck. This is also another image of truck with a auto roof.

[poornimajd]

@AlexeyAB ,is this happening because of the model over generalizing?For example the model may never really see some of the augmentations in real world,so this might be the reason for decrease in accuracy? Any suggestion is appreciated.

[AlexeyAB]

Only 2 reasons:

1. Your labels are wrong, conflicting, mutually exclusive ... https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

check that each object that you want to detect is mandatory labeled in your dataset - no one object in your data set should not be without label. In the most training issues - there are wrong labels in your dataset (got labels by using some conversion script, marked with a third-party tool, ...). Always check your dataset by using: https://github.com/AlexeyAB/Yolo_mark

2. Very small dataset: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

for each object which you want to detect - there must be at least 1 similar object in the Training dataset with about the same: shape, side of object, relative size, angle of rotation, tilt, illumination. So desirable that your training dataset include images with objects at diffrent: scales, rotations, lightings, from different sides, on different backgrounds - you should preferably have 2000 different images for each class or more, and you should train 2000*classes iterations or more

[poornimajd]

Thanks for the detailed explanation.