search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.66k stars 7.96k forks source link

Train Darknet yolov4 voc-based #8139

Open lorenzobattelli opened 2 years ago

lorenzobattelli commented 2 years ago

Hello, I'm new in object detection with Darknet. I'm following the tutorial for object detection on a voc based yolov4 Darknet (https://github.com/Xilinx/Vitis-AI-Tutorials/tree/master/Design_Tutorials/07-yolov4-tutorial#31-darknet-model-training-on-voc) and trying to train the net, but this time using a gtsdb dataset, with the command 

./darknet detector train cfg/voc.data cfg/yolov4.cfg /yolov4.weights -map -dont_show -show_imgs

of course I edited the "voc.data" in order to point the right gtsdb files, I just forgot to rename that file. I edited the cfg files as requested, and the voc.data too. I'm working on a ubuntu VM (god..) I'd need some hints and answers about the training process: 1)After running the train command, should I stop it manually (Ctrl-C) just after I realized the training converged properly, or not? 2)Training convergence in this case means that the loss (or mAP ?) stops decreasing? I used -map parameter but I honestly don't understand where that information is. This is a piece of output:

v3 (iou loss, Normalizer: (iou: 0.07, obj: 1.00, cls: 1.00) Region 133 Avg (IOU: 0.419995), count: 34, class_loss = 3764.068115, iou_loss = 28.210449, total_loss = 3792.278564 
v3 (iou loss, Normalizer: (iou: 0.07, obj: 1.00, cls: 1.00) Region 144 Avg (IOU: 0.253230), count: 5, class_loss = 1026.847412, iou_loss = 0.271484, total_loss = 1027.118896 
v3 (iou loss, Normalizer: (iou: 0.07, obj: 1.00, cls: 1.00) Region 155 Avg (IOU: 0.000000), count: 1, class_loss = 268.681885, iou_loss = 0.000000, total_loss = 268.681885 
 total_bbox = 39, rewritten_bbox = 0.000000 % 

500504: 1169.513062, 1602.209351 avg loss, 0.000013 rate, 15818.413367 seconds, 32032256 images, 2120438.711756 hours left

3) how can I save the weights during training? I bet I have to add something in the command. 3.1) and where can I find those .weights files? in the voc.data file I specified "backup = ./backup" I've let that process run for all night but still I can't see any weight file saved during training. Maybe is it just a matter of time?

4) In the output, which is the number of the current iteration? 4.1) 1 iteration == 1 epoch ?

Thank you for your time

WongKinYiu commented 2 years ago

1 epoch \~= (num_of_train_data / mini_batch_size) iterations