search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.8k stars 7.97k forks source link

Training - refinement during the last batches? #4640

Open mmartin56 opened 4 years ago

mmartin56 commented 4 years ago

Hi Alexey,

Is there a particular reason why the parameter 'max_batches' in yolov2.cfg, yolov3.cfg and yolov3-spp.cfg is set to 500,200 and not 500,000? Is there anything 'special' that happens during these last 200 iterations?

I thought it didn't matter until I tested one of my detector at different training stages:

After 490000 Iterations → got 33.52 mAP@50 After 495000 Iterations → got 34.53 mAP@50 After 500000 Iterations → got 36.26 mAP@50 After 500200 Iterations → got 41.5 mAP@50

As you can see there's a big jump in mAP from 500,000 to 500,200 iterations. It could be just luck, too, but is there anything in the code that could explain that?

Thanks

(Note that this is independent of steps/scales, which I disabled. )

AlexeyAB commented 4 years ago

It is related only for original repo, where the last 200 iterations are used for fixed 608x608 network resolution, while previous iterations are trained for random network sizes: https://github.com/pjreddie/darknet/blob/61c9d02ec461e30d55762ec7669d6a1d3c356fb2/examples/detector.c#L66

mmartin56 commented 4 years ago

Thanks @AlexeyAB. I use your repo so it can't be that. Turns out there is quite a lot of variance in what detector gets output during training. It's probably better to actually use steps/scales.