Benchmarking YOLO Versions for Custom Object Detection Task

Ekanshh commented 4 days ago

Search before asking

[X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

There are multiple versions of YOLO (e.g., YOLOv8x, YOLOv9x, YOLOv10x). Is there an easy way to benchmark these models for an object detection task on a custom dataset, so I can select the one that works best for my application? If not, what is the recommended way to perform such benchmarking?

Additional

No response

github-actions[bot] commented 4 days ago

👋 Hello @Ekanshh, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

glenn-jocher commented 4 days ago

@Ekanshh hello,

Thank you for your question! Benchmarking different YOLO versions on your custom dataset is a great approach to determine the best model for your specific application. Fortunately, Ultralytics provides a straightforward way to benchmark models using the benchmark function.

Here's a concise guide on how you can benchmark different YOLO models:

Install the Latest Version: Ensure you have the latest version of the Ultralytics package installed. You can update it using:
```
pip install -U ultralytics
```
Prepare Your Dataset: Make sure your custom dataset is properly formatted and ready for training and validation. You can follow the dataset preparation guidelines here.

Benchmarking Script: You can use the following Python script to benchmark different YOLO models. This script will evaluate the models on your custom dataset and provide metrics such as mAP and inference time.

from ultralytics import YOLO
from ultralytics.utils.benchmarks import benchmark

# List of models to benchmark
models = ['yolov8x.pt', 'yolov9x.pt', 'yolov10x.pt']

# Path to your custom dataset
data_path = 'path/to/your/custom_dataset.yaml'

# Benchmark each model
for model_name in models:
    print(f"Benchmarking {model_name}...")
    model = YOLO(model_name)
    results = benchmark(model=model, data=data_path, imgsz=640, half=False, device=0)
    print(results)

Analyze Results: The benchmark function will return detailed metrics for each model, allowing you to compare their performance and select the best one for your application.

If you encounter any issues during the benchmarking process, please ensure you provide a reproducible example as outlined here. This will help us assist you more effectively.

Feel free to reach out if you have any further questions or need additional assistance. Happy benchmarking! 🚀

ultralytics / ultralytics