Validating YOLO on custom dataset

[ghadahamed]

First of all, thanks very much, the guide helped me a lot.. Thanks for all your efforts,

I am a PhD student that is working of breast cancer detection. I follow your guide to help me detect and classify masses. So, I have several questions, kindly can you help me @AlexeyAB 1- If I worked on small dataset for mammograms contains 410 images, shall this is considered a possible number of samples to train with? if NO what is the minimum number of samples shall be available for training YOLO on. 2- If I can train with 410 mammograms, what are the best values for the learning rate and the batches to get beneficial training? or any other parameters shall I set to optimize IOU and the overall results. 3- Why results of the same batch change each time the model trained? 4- You told that if nan appears in some lines, then it is okay, at the first subdivisions per the first batch a lot of nans appear followed by values, then in the second batch the output is as follows: So, is this okay? (in other runs, the nan values decreased than this). So kindly can you guide me if this okay or not? 5- In batch#4: the avg loss = 2006.245117 and in batch#3: the avg loss = 2006.455688, here in batch 4 decreased than 3 but sometimes increased (this is occurs in the first running batches). Is this okay or there is something wrong? 6- The rate always equal to zero, however i leave the learning rate value as it is in the main code, how can maintain this? 7- If i need to accelerate the training process what shall I do using this dataset?

Thanks in advance for your help and time and waiting for your help @AlexeyAB

[AlexeyAB]

@ghadahamed Hi,

1. You can use any number of images. In some cases 400 images are enough. Always, more images - better result.
   • The general rule: for each object which you want to detect - there must be at least 1 similar object in the Training dataset with about the same: shape, side of object, relative size, angle of rotation, tilt, illumination.
   • The simplest rule: you should use at least 2000 images per class

https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

for each object which you want to detect - there must be at least 1 similar object in the Training dataset with about the same: shape, side of object, relative size, angle of rotation, tilt, illumination. So desirable that your training dataset include images with objects at diffrent: scales, rotations, lightings, from different sides, on different backgrounds - you should preferably have 2000 different images for each class or more, and you should train 2000*classes iterations or more

---

2. Use default learning_rate and batch. You just should train at least 4000 iterations or more.

---

3- Why results of the same batch change each time the model trained?

3. What do you mean? What is the result? During training each image randomly changed
   • changed aspect ration (parameter: jitter in [yolo])
   • changed size (parameters: jitter & random in [yolo])
   • changed color (parameters: hue, saturation, exposure in [net])
   • randomly horizontal flipped (parameters: flip=1 in [net] by default)

---

4. This is OK. Do you use GPU for training? Without GPU you will train for all your life.

---

5. This is OK. The Training Loss always fluctuates.

---

6. For the first 1000 iterations learning rate will increase exponentially from 0 to 0.001 (because burn_in=1000 learning_rate=0.001 in cfg-file), it gives the best result in the long run.

7. To accelerate the training, just use any nVidia GPU and build it with GPU=1 CUDNN=1 in the Makefile (or darknet.sln on Windows).

[ghadahamed]

@AlexeyAB thanks a lot for your continuous support and help, your answers really helped me. Kindly can I ask if I want to change the number of epochs and the number of iterations inside each epoch. What shall I do?

[AlexeyAB]

@ghadahamed

• You can change max iterations for training = max_batches in cfg-file

• number of iterations inside epoch = number_of_training_images / batch_in_cfg

[ghadahamed]

Okay will do this. Thanks a lot @AlexeyAB