How to use a cfg?

[ekarabulut]

Hi,

I've been using Darknet (Alexey) for a while and I am familiar with how to configure a cfg for a custom training for object detection. This repository is suggested to me and the claimed results seem promising.

But, I could not find how to use a cfg for a training. For instance, I could not figure out how to adapt the cfg to number of classes I'd like to train for. In regular Darknet, one has to configure some parameters in Yolo layer (number of filters, classes etc) in order to adapt the cfg to a custom training.

Any help is appreciated

[AlexeyAB]

@ekarabulut Hi,

Do the same thing: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

[ekarabulut]

Hi

Thank you for the reply. In Darknet almost all configuration related to number of classes happen in the Yolo layers and Convolutional layers before a Yolo layer. In - for example - CSPDarknet53 or CSPDarknet53-Elastic cfg files, how can I set number of classes in such cfg files?

[AlexeyAB]

CSPDarknet53 or CSPDarknet53-Elastic

These models are Classifiers for ImageNet dataset.

---

Look at Detectors for MS COCO dataset: https://github.com/WongKinYiu/CrossStagePartialNetworks#ms-coco

Or these models: https://github.com/WongKinYiu/CrossStagePartialNetworks/tree/master/in%20progress

[WongKinYiu]

sorry, wrong click.

[Rajasekhar06]

Using instructions for custom object detections using yolov3 instructions throws an error like this

./darknet detector train data/yolov3-spp-3cls.data cfg/csresnext50-panet-spp.cfg -gpus 0,1,2,3

0,1,2,3 csresnext50-panet-spp compute_capability = 750, cudnn_half = 1 net.optimized_memory = 0 batch = 4, time_steps = 1, train = 1 layer filters size/strd(dil) input output 0 conv 64 7 x 7/ 2 608 x 608 x 3 -> 304 x 304 x 64 1.739 BF 1 max 2x 2/ 2 304 x 304 x 64 -> 152 x 152 x 64 0.006 BF 2 conv 128 1 x 1/ 1 152 x 152 x 64 -> 152 x 152 x 128 0.379 BF 3 route 1 -> 152 x 152 x 64 4 conv 64 1 x 1/ 1 152 x 152 x 64 -> 152 x 152 x 64 0.189 BF 5 conv 128 1 x 1/ 1 152 x 152 x 64 -> 152 x 152 x 128 0.379 BF 6 conv 128/ 32 3 x 3/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.213 BF 7 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 8 Shortcut Layer: 4 w = 152, w2 = 152, h = 152, h2 = 152, c = 128, c2 = 64 9 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 10 conv 128/ 32 3 x 3/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.213 BF 11 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 12 Shortcut Layer: 8 13 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 14 conv 128/ 32 3 x 3/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.213 BF 15 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 16 Shortcut Layer: 12 17 conv 128 1 x 1/ 1 152 x 152 x 128 -> 152 x 152 x 128 0.757 BF 18 route 17 2 -> 152 x 152 x 256 19 conv 256 1 x 1/ 1 152 x 152 x 256 -> 152 x 152 x 256 3.028 BF 20 conv 256/ 32 3 x 3/ 2 152 x 152 x 256 -> 76 x 76 x 256 0.213 BF 21 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 22 route 20 -> 76 x 76 x 256 23 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 24 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 25 conv 256/ 32 3 x 3/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.213 BF 26 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 27 Shortcut Layer: 23 28 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 29 conv 256/ 32 3 x 3/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.213 BF 30 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 31 Shortcut Layer: 27 32 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 33 conv 256/ 32 3 x 3/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.213 BF 34 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 35 Shortcut Layer: 31 36 conv 256 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 256 0.757 BF 37 route 36 21 -> 76 x 76 x 512 38 conv 512 1 x 1/ 1 76 x 76 x 512 -> 76 x 76 x 512 3.028 BF 39 conv 512/ 32 3 x 3/ 2 76 x 76 x 512 -> 38 x 38 x 512 0.213 BF 40 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 41 route 39 -> 38 x 38 x 512 42 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 43 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 44 conv 512/ 32 3 x 3/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.213 BF 45 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 46 Shortcut Layer: 42 47 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 48 conv 512/ 32 3 x 3/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.213 BF 49 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 50 Shortcut Layer: 46 51 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 52 conv 512/ 32 3 x 3/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.213 BF 53 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 54 Shortcut Layer: 50 55 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 56 conv 512/ 32 3 x 3/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.213 BF 57 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 58 Shortcut Layer: 54 59 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 60 conv 512/ 32 3 x 3/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.213 BF 61 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 62 Shortcut Layer: 58 63 conv 512 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 512 0.757 BF 64 route 63 40 -> 38 x 38 x1024 65 conv 1024 1 x 1/ 1 38 x 38 x1024 -> 38 x 38 x1024 3.028 BF 66 conv 1024/ 32 3 x 3/ 2 38 x 38 x1024 -> 19 x 19 x1024 0.213 BF 67 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 68 route 66 -> 19 x 19 x1024 69 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 70 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 71 conv 1024/ 32 3 x 3/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.213 BF 72 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 73 Shortcut Layer: 69 74 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 75 conv 1024/ 32 3 x 3/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.213 BF 76 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 77 Shortcut Layer: 73 78 conv 1024 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x1024 0.757 BF 79 route 78 67 -> 19 x 19 x2048 80 conv 2048 1 x 1/ 1 19 x 19 x2048 -> 19 x 19 x2048 3.028 BF 81 conv 512 1 x 1/ 1 19 x 19 x2048 -> 19 x 19 x 512 0.757 BF 82 conv 1024 3 x 3/ 1 19 x 19 x 512 -> 19 x 19 x1024 3.407 BF 83 conv 512 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x 512 0.379 BF 84 max 5x 5/ 1 19 x 19 x 512 -> 19 x 19 x 512 0.005 BF 85 route 83 -> 19 x 19 x 512 86 max 9x 9/ 1 19 x 19 x 512 -> 19 x 19 x 512 0.015 BF 87 route 83 -> 19 x 19 x 512 88 max 13x13/ 1 19 x 19 x 512 -> 19 x 19 x 512 0.031 BF 89 route 88 86 84 83 -> 19 x 19 x2048 90 conv 512 1 x 1/ 1 19 x 19 x2048 -> 19 x 19 x 512 0.757 BF 91 conv 1024 3 x 3/ 1 19 x 19 x 512 -> 19 x 19 x1024 3.407 BF 92 conv 512 1 x 1/ 1 19 x 19 x1024 -> 19 x 19 x 512 0.379 BF 93 conv 256 1 x 1/ 1 19 x 19 x 512 -> 19 x 19 x 256 0.095 BF 94 upsample 2x 19 x 19 x 256 -> 38 x 38 x 256 95 route 65 -> 38 x 38 x1024 96 conv 256 1 x 1/ 1 38 x 38 x1024 -> 38 x 38 x 256 0.757 BF 97 route 96 94 -> 38 x 38 x 512 98 conv 256 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 256 0.379 BF 99 conv 512 3 x 3/ 1 38 x 38 x 256 -> 38 x 38 x 512 3.407 BF 100 conv 256 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 256 0.379 BF 101 conv 512 3 x 3/ 1 38 x 38 x 256 -> 38 x 38 x 512 3.407 BF 102 conv 256 1 x 1/ 1 38 x 38 x 512 -> 38 x 38 x 256 0.379 BF 103 conv 128 1 x 1/ 1 38 x 38 x 256 -> 38 x 38 x 128 0.095 BF 104 upsample 2x 38 x 38 x 128 -> 76 x 76 x 128 105 route 38 -> 76 x 76 x 512 106 conv 128 1 x 1/ 1 76 x 76 x 512 -> 76 x 76 x 128 0.757 BF 107 route 106 104 -> 76 x 76 x 256 108 conv 128 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 128 0.379 BF 109 conv 256 3 x 3/ 1 76 x 76 x 128 -> 76 x 76 x 256 3.407 BF 110 conv 128 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 128 0.379 BF 111 conv 256 3 x 3/ 1 76 x 76 x 128 -> 76 x 76 x 256 3.407 BF 112 conv 128 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 128 0.379 BF 113 conv 256 3 x 3/ 1 76 x 76 x 128 -> 76 x 76 x 256 3.407 BF 114 conv 21 1 x 1/ 1 76 x 76 x 256 -> 76 x 76 x 21 0.062 BF 115 yolo

Error: l.outputs == params.inputs filters= in the [convolutional]-layer doesn't correspond to classes= or mask= in [yolo]-layer

How to solve this? like how many filters are to be set for new network I made changes as per older method like filters and classes and steps but it is different for this!

[WongKinYiu]

@Rajasekhar06

plz follow https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

according to ur .data file name "yolov3-spp-3cls.data", i assume there r 3 classes in ur dataset. so the number of filter of convolutional layers which r just before yolo layers should be ((classes + 5)x3) = ((3 + 5)x3) = 24, however, it is set to 21 in ur cfg file.

[Rajasekhar06]

@Rajasekhar06

plz follow https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

according to ur .data file name "yolov3-spp-3cls.data", i assume there r 3 classes in ur dataset. so the number of filter of convolutional layers which r just before yolo layers should be ((classes + 5)x3) = ((3 + 5)x3) = 24, however, it is set to 21 in ur cfg file.

Thanks So much I made this change so many times but this time I forgot to do that its working

[Rajasekhar06]

@WongKinYiu I have another small query How to set max batches in multi gpu training on single gpu we do that multiplying class with 2000 but the README.md is a little confusing for steps, max_batches

[WongKinYiu]

@Rajasekhar06

becuz it is depend on ur dataset size and task difficulty. usually, we set it near to 300*num_of_images/batch_size.

[Rajasekhar06]

@WongKinYiu I have completed training by setting old style of max batched like max_batches=2000*No. of classes which is suited for Single GPU training with yolov3-spp config after 6000 iterations then avg.loss was 0.7xx but this time I used Multiple GPUs(4 x TITAN RTX 24GB each) and your Config file CSPnet where avg.loss was above 1.xxx after 6000 iterations but burn_in=41000=4000(Might be burn_in is the reason for not decreasing avg.loss as i have set this parameter as per multi-GPU but not the iterations as below) for 3 classes dataset, so I re-started training with this `2000num.of GPUsnum.of classesparameters and respective step sizes (80% and 90%) from the last checkpoint results are yet to come I will update you once after training completes. if this did not work then I will set the parameters defined by you300num_of_images/batch_sizeand train.(BTW for multi-GPU what about Burn-In currently I set it tonum.of GPUs * 1000` which blocks the use of tensor cores till half of the max batches)

Thanks for your time and effort :100:

[shashwat1998]

@WongKinYiu can these cfg files be used in pytorch models?

[WongKinYiu]

yes.

[shashwat1998]

yes.

Can you please give some details on how to do it too?

[WongKinYiu]

here provide some examples.

[shashwat1998]

here provide some examples.

Thank You!