Retrain the trained weights

[Juuustin]

it is recommended that: For example, after 2000 iterations you can stop training, and later just start training using: darknet.exe detector train data/obj.data yolo-obj.cfg backup\yolo-obj_2000.weights

I finished the training and I want to continue training longer, so I use ./darknet detector train data/obj.data yolo-obj.cfg backup/yolo-obj_10000.weights -map

Then, the figure occures and it disappered suddenly: [yolo] params: iou loss: ciou (4), iou_norm: 0.07, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.05 nms_kind: greedynms (1), beta = 0.600000 Total BFLOPS 59.563 avg_outputs = 489778 Allocate additional workspace_size = 49.84 MB Loading weights from backup/yolo-obj_10000.weights... seen 64, trained: 640 K-images (10 Kilo-batches_64) Done! Loaded 162 layers from weights-file Learning Rate: 0.001, Momentum: 0.949, Decay: 0.0005 Detection layer: 139 - type = 28 Detection layer: 150 - type = 28 Detection layer: 161 - type = 28 If error occurs - run training with flag: -dont_show Saving weights to backup//yolo-obj_final.weights Create 6 permanent cpu-threads If you want to train from the beginning, then use flag in the end of training command: -clear

[YZAlex]

I have the same problem, have you figured it out? @Juuustin

[stephanecharette]

Did you increase max_batches to continue the training? And if not, did you add -clear 1 at the end of the command so it restarts at iteration zero?

[Juuustin]

I have the same problem, have you figured it out? @Juuustin

Not yet

[Juuustin]

Did you increase max_batches to continue the training? And if not, did you add -clear 1 at the end of the command so it restarts at iteration zero?

Yes, I increased the max_batches, and I want to continue training with the pre-trained weights, don't want to start from zero.

[harsco-jfernandez]

hmm.. I think you can continue training the same model - which may not yield better results - but you can't retrain it unless you dump some number of layers from the bottom and retrain using generated pre-trained weights

[Juuustin]

hmm.. I think you can continue training the same model - which may not yield better results - but you can't retrain it unless you dump some number of layers from the bottom and retrain using generated pre-trained weights

Hi, I mean for example, at first I trained for 1000 batches, after finish training, I want to train 500 more batches, could I start form the 1000-batches weights? If so, how should I do? Hope this example is more clear:)

[harsco-jfernandez]

Yes, you can. You can always continue training. Suppose your machine crashes, it would be unreasonable to have to start from the beginning, and instead, you can continue from where you left off which can be anywhere.

./darknet detector train file.data file.cfg file_1000.weight -gpus 0 -map