Training Error:Model Doesn't Show Results After Training

[Adeel-Intizar]

After Training YOLOv4 on custom dataset consisting 3 different classes, it doesn't show any detections at all. I thought i might be having problem with dataset so i changed it, but still it doesn't show any detections, doesn't matter if i train for 25 epochs or 100. Here is my Script which i used in Colab.. Can you help me with this?

!pip install yolov4
!unzip yolo-dataset.zip

from tensorflow.keras import callbacks, optimizers
from yolov4.tf import SaveWeightsCallback, YOLOv4
import time

yolo = YOLOv4()
yolo.classes = "classes.names"
yolo.input_size = 640
yolo.batch_size = 4

yolo.make_model()
yolo.load_weights(
    "/content/drive/My Drive/Colab Notebooks/yolo weights/yolov4.conv.137",
    weights_type="yolo"
)

train_data_set = yolo.load_dataset(
    "yolo_train.txt",
    image_path_prefix="yolo-animal-detection-small/train",
    label_smoothing=0.05
)
val_data_set = yolo.load_dataset(
    "yolo_val.txt",
    image_path_prefix="yolo-animal-detection-small/test",
    training=False
)

epochs = 25
lr = 1e-4

optimizer = optimizers.Adam(learning_rate=lr)
yolo.compile(optimizer=optimizer, loss_iou_type="ciou", loss_verbose=0)

def lr_scheduler(epoch):
    if epoch < int(epochs * 0.6):
        return lr
    if epoch < int(epochs * 0.8):
        return lr * 0.5
    if epoch < int(epochs * 0.9):
        return lr * 0.1
    return lr * 0.01

_callbacks = [
    callbacks.LearningRateScheduler(lr_scheduler),
    callbacks.TerminateOnNaN(),
    callbacks.TensorBoard(
        log_dir="logs",
    ),
    SaveWeightsCallback(
        yolo=yolo, dir_path="trained",
        weights_type="yolo", epoch_per_save=1
    ),
]

yolo.fit(
    train_data_set,
    epochs=epochs,
    callbacks=_callbacks,
    validation_data=val_data_set,
    validation_steps=1,
    validation_freq=1,
    steps_per_epoch=100,
)

from yolov4.tf import YOLOv4

model = YOLOv4()

model.classes = "classes.names"
model.input_size = 640

model.make_model()
model.load_weights("trained/yolov4-final.weights", weights_type="yolo")

from PIL import Image
model.inference('yolo-animal-detection-small/test/cats_000.jpg')
Image.open('out.jpg')

Here are some of the files... yolo_train.txt yolo_val.txt

[hhk7734]

How to get out.jpg?

[hhk7734]

https://github.com/hhk7734/tensorflow-yolov4/blob/25e802f1e0b95a5719920e40e98fdac1d0a9bfb2/py_src/yolov4/common/base_class.py#L246-L261

Try the below code

...
model.make_model()
model.load_weights("trained/yolov4-final.weights", weights_type="yolo")

import cv2
from google.colab.patches import cv2_imshow

image = cv2.imread('yolo-animal-detection-small/test/cats_000.jpg')
frame = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) 
bboxes = model.predict(frame)
image = self.draw_bboxes(image, bboxes) 
cv2_imshow(image) 

[Adeel-Intizar]

How to get out.jpg?

I modified cv2.imshow with cv2.imwrite

[Adeel-Intizar]

https://github.com/hhk7734/tensorflow-yolov4/blob/25e802f1e0b95a5719920e40e98fdac1d0a9bfb2/py_src/yolov4/common/base_class.py#L246-L261

Try the below code

...
model.make_model()
model.load_weights("trained/yolov4-final.weights", weights_type="yolo")

import cv2
from google.colab.patches import cv2_imshow

image = cv2.imread('yolo-animal-detection-small/test/cats_000.jpg')
frame = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) 
bboxes = model.predict(frame)
image = self.draw_bboxes(image, bboxes) 
cv2_imshow(image) 

Still No Result... No Output is Shown

[hhk7734]

If there are no other problems, it seems to be a training failure.

[hhk7734]

I have a lot of work right now, so it's hard to test it right now.

[Adeel-Intizar]

I hope you fix this Error as soon as possible, i look forward to it. Besides, i have attached the dataset i used, i don't see any problem with dataset, but maybe you would like to have a look at the dataset. yolo-dataset.zip

[Adeel-Intizar]

@hhk7734 Did you fix it?

[hhk7734]

Not yet tested. Can you try this script? https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-dataset/#dataset-test-script

[Adeel-Intizar]

I have one more question, other than 'converted_coco' format how can I use 'yolo' format in your implementation to load dataset?

[hhk7734]

Ref: https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-dataset#dataset-files-and-formats

It looks like yolo_train.txt is wrong. Please check the file format.

[Adeel-Intizar]

Thanks, I found out that dataset annotation is not right

[Adeel-Intizar]

@hhk7734 i think i am asking too much questions here, but this dataset is well annotated with respect to bounding boxes, and your test script shows bounding boxes on images, but still model doesn't show results on this too, i have attached dataset and colab notebook link, please have a look yolo-clothing-data.zip

https://colab.research.google.com/drive/1v7TKtrut1sVxyBgRjPrievQ_AI_x-5fO?usp=sharing

[hhk7734]

Your batch_size * step * epoch is 2 * 100 * 10. The number of classes is 9.

On my test batch_size * step * epoch is 8 * 100 * 60. The number of classes is 3.

yolov4.conv.137 is made from coco2017 dataset.

On my test, my classes are related to coco classes. but yours are not. You may need more datasets and more epochs.

When I tested on your dataset, tiny=True batch_size = 8 input_size = 416 epochs = 20 lr = 1e-4 Removed callbacks.LearningRateScheduler(lr_scheduler) from _callbacks Detected some classes with low probability.

The reason to use tiny is just to shorten the test time. you don't need to set tiny=True. On colab, using my dataset, the training took about 80 minutes for 3 classes.

[Adeel-Intizar]

Can you share your dataset and script?

[hhk7734]

I made a dataset from coco-2017. Ref: https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-dataset#convert-coco-to-custom-dataset

Download : https://cocodataset.org/#download 2017 Train images, 2017 Val images, 2017 Train/Val annotations.

Script: https://colab.research.google.com/drive/16Qzm3cElQ0J8-xFPfhAq-ZE11gpb35zD?usp=sharing

[Adeel-Intizar]

colab notebook needs access

[hhk7734]

Done. :)

[Adeel-Intizar]

Thanks for your help, I started to use "yolo" format instead of "converted_coco" and it is working