Custom Backbone Implementation

[ragz4125]

Discussed in https://github.com/ultralytics/yolov5/discussions/11874

^{Originally posted by **ragz4125** July 17, 2023} I want to use another architecture instead of darknet to get features from input Can anyone tell my in what all files the changes has to be made in step by step manner? PS: I am trying to use it for object detection Thank you in advance for any help

[github-actions[bot]]

👋 Hello @ragz4125, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@ragz4125, thank you for reaching out 😊! To use a custom backbone architecture instead of Darknet in YOLOv5 for object detection, you'll need to make changes in the following files:

1. models/yolo.py: This is the main file where the YOLO architecture is defined. You can replace the backbone network defined in the __init__ method with your custom backbone implementation.

2. models/common.py: If your custom backbone architecture requires any additional blocks or functions, you can define them in this file, inside the Focus, Conv, or other relevant classes.

3. models/yolo.py (again): Modify the forward method to incorporate your custom backbone architecture. Ensure that the output of the backbone network is passed through the subsequent YOLO layers correctly.

Remember to thoroughly test your changes to ensure that the model is still functioning correctly. Let us know if you encounter any difficulties or if you need further assistance. Good luck with your custom backbone implementation!

[ragz4125]

@glenn-jocher Thank you for the reply and in 1st modification you mentioned on init() you meant we have to replace the given yaml file with modified yaml file right? And information on what lines of code in yolo.py have to be modified will be more helpful!!

[glenn-jocher]

@ragz4125 thank you for your response! I apologize for any confusion caused. In the __init__() method in models/yolo.py, you actually need to modify the backbone network architecture defined within the method. You can replace the existing backbone with your custom implementation.

As for the specific lines of code in yolo.py that need to be modified, it depends on the structure of your custom backbone architecture. Generally, you will need to replace or modify the lines that define the backbone layers, such as convolutional layers, upsampling layers, or any other operations specific to your architecture.

If you have any further questions or need more assistance, please let me know.

[ragz4125]

@glenn-jocher Every class has init() According to my understanding in most of the init() the yaml file is passed so I think the yaml file have to be modified according to out architecture please correct me if I am wrong

[glenn-jocher]

Hi @! You are correct that the __init__() method is usually where the YAML file is passed in many classes. In the context of the YOLOv5 repository, the YAML file typically contains configuration settings for the model architecture and training. However, in the case of modifying the backbone architecture, you would directly modify the code within the __init__() method itself.

The YAML file is not directly involved in replacing the backbone architecture. Instead, you would replace the existing backbone network implementation with your custom one in the __init__() method within the respective files. If you have any further questions or need more clarity, feel free to ask.

[ragz4125]

@glenn-jocher Where do we find the neck part(i.e the feature from backbone gets passed to head) in source-code?

[glenn-jocher]

@ragz4125 the "neck" part in YOLOv5, where the features from the backbone network are passed to the head, is not explicitly defined in the source code as a separate module. Instead, it is integrated within the architecture design of the YOLOv5 model.

In YOLOv5, after passing through the backbone network, the features go through a series of additional convolutional layers before being passed to the detection head. These convolutional layers help to further refine and abstract the features before they are used for object detection.

You can find the code related to this feature refinement and integration in the models/yolo.py file. Specifically, check the forward() method where the features are processed through convolutions and other operations to prepare them for the detection head.

If you need more specific details or have further questions, please let me know.

[Mahfuzkiron]

Hello, I need some help. My dataset is imbalanced and I am trying to use focal loss with class weights in the yolov8 model but when I run this code output doesn't change that means without focal loss and weight key the same output as yolov8. This is the ref code. Please help me to successfully to solve this problem. I have been trying long time but i can not solve it. thank you in advance.

Import necessary libraries

import os from ultralytics import YOLO import torch import torch.nn as nn

Define dataset paths

train_images_path = '.../train/images' train_labels_path = '.../train/labels' val_images_path = '.../val/images' val_labels_path = '.../val/labels' data_yaml_path = '.../data.yaml'

Create a data.yaml file for YOLOv8

data_yaml_content = f""" train: {train_images_path} # path to training images val: {val_images_path} # path to validation images

nc: 7 # number of classes names: ['breakage','contamination','crack', 'dirt','good','missing','shelter'] # class names """

Save the data.yaml file

with open(data_yaml_path, 'w') as f: f.write(data_yaml_content)

Define custom Focal Loss function

class FocalLoss(nn.Module): def init(self, alpha=0.25, gamma=2.0, reduction='mean'): super(FocalLoss, self).init() self.alpha = alpha self.gamma = gamma self.reduction = reduction

def forward(self, inputs, targets):
    BCE_loss = nn.CrossEntropyLoss(reduction='none')(inputs, targets)
    pt = torch.exp(-BCE_loss)
    F_loss = self.alpha * (1 - pt) ** self.gamma * BCE_loss
    return F_loss.mean() if self.reduction == 'mean' else F_loss.sum()

Load the YOLOv8 model and replace default loss with custom Focal Loss

model = YOLO('/custom_yolov8n.yaml') # Load YOLOv8 model

Define class weights (higher for defect classes)

class_weights = torch.tensor([3.0, 2.5, 3.5, 3.5, 1.0, 2.5, 2.5], dtype=torch.float32)

Update model with custom loss and class weights

model.model.model.loss_obj = FocalLoss() model.model.model.loss_cls = nn.CrossEntropyLoss(weight=class_weights)

Training parameters

model.train( data=data_yaml_path, # Path to your YAML data configuration imgsz=128, # Image size batch=64, # Batch size epochs=50, # Number of epochs lr0=0.01, # Initial learning rate lrf=0.1, # Final learning rate weight_decay=0.0005, # Weight decay momentum=0.937, # Momentum optimizer='SGD', # Optimizer save_period=10, # Save every n epochs cache="disk" # Cache images for faster training )

Evaluate the model after training

results = model.val() # Validate the model on the validation set print(results) # Print validation results

[pderrenger]

It seems like you're trying to implement focal loss and class weights in YOLOv8, but the changes aren't reflecting in your results. Ensure that your custom loss functions are correctly integrated into the model's training loop. Also, verify that the model is using your custom loss functions by checking the training logs for any discrepancies. If the issue persists, consider testing with a smaller dataset to debug more effectively.