search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.65k stars 7.96k forks source link

About Mixup #4446

Open May-forever opened 4 years ago

May-forever commented 4 years ago

Hi, @AlexeyAB .

How to use mixup in training networks?

How to see the changes in training images and save them?

AlexeyAB commented 4 years ago

Just set in cfg-file:

[net]
mixup=1

How to see the changes in training images and save them?

Use flag -show_imgs at the end of training command ./darknet detector train ... -show_imgs it will show and save augmented images near with executable file

if you don't want to show them (just for saving jpg files) use: ./darknet detector train ... -show_imgs -dont_show

For Classifier there are:

For Detector there are:

May-forever commented 4 years ago

Just set in cfg-file:

[net]
mixup=1

How to see the changes in training images and save them?

Use flag -show_imgs at the end of training command ./darknet detector train ... -show_imgs it will show and save augmented images near with executable file

if you don't want to show them (just for saving jpg files) use: ./darknet detector train ... -show_imgs -dont_show

For Classifier there are:

  • mixup=1 #3272
  • cutmix=1 #4419
  • mosaic=1 #4432

For Detector there are:

  • mixup=1 #3272
  • mosaic=1 #4264

Hi @AlexeyAB , thank you very much for your help.

Can't cutmix=1 be used to train a detector yet?

Looking forward to hearing from you, thanks a lot.

nyj-ocean commented 4 years ago

@AlexeyAB Can I set mixup=1 and mosaic=1 together in yolov3.cfg or Gaussian+CIoU+yolov3.cfg to improve mAP?

[net]
batch=32
subdivisions=16
width=416
height=416
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1
mixup=1
mosaic=1
AlexeyAB commented 4 years ago

@nyj-ocean

If you set both - then will be used only Mosaic

mixup=1
mosaic=1

Because in the most tests MixUp doesn't give any gain in AP during training Detector (MixUp gives boost of accuracy only for training Classifier with mixup=1, and then this pre-trained weights file with higher Top1 is used for training Detector with mixup=0 - so it gives higher ~+4 AP)

Mosaic should increase accuracy for both training Classifier and Detector.

So is recommended:

nyj-ocean commented 4 years ago

@AlexeyAB If I only set mosaic=1 in yolov3.cfg. Then can I directly use the pre-weights darknet53.conv.74 to train mosaic+yolov3?

Or should I train Classifier with mosaic=1 at first,and then this pre-trained weights file with higher Top1 is used for training Detector with mosaic=1?

AlexeyAB commented 4 years ago

Or should I train Classifier with mosaic=1 at first,and then this pre-trained weights file with higher Top1 is used for training Detector with mosaic=1?

Yes, you can. And it will increase accuracy. But for an even greater increase in accuracy, first train the Classifier with a mosaic and cutmix, and then the Detector with a mosaic.

nyj-ocean commented 4 years ago

@AlexeyAB I try mosaic+yolov3 in the latest repo. I only set mosaic=1 in yolov3.cfg.Then I directly use my own dataset to train mosaic+yolov3 with the pre-weights darknet53.conv.74. It can train normally at the beginning ,but produce NAN later as following. chart

But I can train mixup+yolov3 normally,and it do not produce NAN

AlexeyAB commented 4 years ago

@nyj-ocean

nyj-ocean commented 4 years ago

@AlexeyAB

GPU=1 CUDNN=1 CUDNN_HALF=1 OPENCV=1 AVX=0 OPENMP=0 LIBSO=0 ZED_CAMERA=0

nyj-ocean commented 4 years ago

@AlexeyAB But I can train mixup+yolov3 normally,and it do not produce NAN as following mixup-1-chart

chart

nyj-ocean commented 4 years ago

@AlexeyAB mosaic+yolov3 produce Nan about 20k steps as shown in https://github.com/AlexeyAB/darknet/issues/4446#issuecomment-562798214 So I use the mosaic+yolov3_16000.weigts in backup file to continue training. But it produce Nan again about 25k steps as shown following (it breaks at 24k steps ) sic-continu-1-chart sic-continue-2-chart

AlexeyAB commented 4 years ago

@nyj-ocean

I improved mosaic=1 for Detector. Try new mosaic. Two commits: https://github.com/AlexeyAB/darknet/commit/87f36b79f54ac0b84512a3c1c4284d31d100f6f7 and https://github.com/AlexeyAB/darknet/commit/13f064f1be1ccf28fae002349ba53fbf61a122a5

nyj-ocean commented 4 years ago

@AlexeyAB Thanks. By the way, can I set mosaic=1 and activation=swish in yolov3.cfg ,and train with darknet53.conv.74 to further improve mAP like following?

[net] batch=32 subdivisions=16 width=416 height=416 channels=3 momentum=0.9 decay=0.0005 angle=0 saturation = 1.5 exposure = 1.5 hue=.1 mosaic=1 ... ... [convolutional] batch_normalize=1 filters=32 size=3 stride=1 pad=1 activation=swish

nyj-ocean commented 4 years ago

@AlexeyAB I only replace activation=swish with activation=leaky in yolov3.cfg,and use darknet53.conv.74 to train. It do not get any improvement in mAP, but decrease a bit.

AlexeyAB commented 4 years ago

@nyj-ocean darknet53.conv.74 is trained with leaky-relu. You should retrain darknet53 classifier with swish.

Re-train Darknet53.cfg classifier with activation=swish and mosaic=1 cutmix=1 on ImageNet then do partial /.darknet partial cfg/darknet53_448.cfg darknet53_448.weights darknet53.conv.74 74

and use this new pre-trained weights darknet53.conv.74 for training detector with activation=swish and mosaic=1

nyj-ocean commented 4 years ago

@AlexeyAB I set mixup=1 in yolov3.cfg, and use my own dataset to train with darknet53.conv.74.

After training, I use the best.weights in the backup file to calculate mAP with the following command:

./darknet detector map my_obj.data mixup+yolov3.cfg backup/best.weights -points 0

It gets a 0.6% improvement in mAP

I am not sure this improvement in mAP is just for random or for the mixup=1

AlexeyAB commented 4 years ago

@nyj-ocean In such cases, they usually train 3 times with mixup=1 and 3 times without, and take average accuracy.

But instead I suggest to train Detector with new mosaic=1

nyj-ocean commented 4 years ago

@AlexeyAB I will try to set mosaic=1 in yolov3.cfg, and train with darknet53.conv.74 in latest repo.

nyj-ocean commented 4 years ago

@AlexeyAB I set mosaic=1 in yolov3.cfg, and train with darknet53.conv.74 in latest repo. (dataset: 1500 images for training set ,other 1500 images for val set) After training, I use the best.weights in the backup file to calculate mAP with the following command:

./darknet detector map my_obj.data mosaic+yolov3.cfg backup/best.weights -points 0

It gets a 0.2% improvement in mAP compared with yolov3

mixup+yolov3 gets a 0.6% improvement in mAP compared with yolov3 as shown in https://github.com/AlexeyAB/darknet/issues/4446#issuecomment-562935289

AlexeyAB commented 4 years ago

@nyj-ocean Do you use the latest version of Darknet?

nyj-ocean commented 4 years ago

@AlexeyAB https://github.com/AlexeyAB/darknet/issues/4446#issuecomment-562856479

@nyj-ocean I improved mosaic=1 for Detector. Try new mosaic. Two commits: 87f36b7 and 13f064f

I download this repo and start to train mosaic+yolov3 just after when you improved mosaic=1 https://github.com/AlexeyAB/darknet/issues/4446#issuecomment-562856479