Ensembling two models

[AmmarOkran]

❔Question

Hi, I have a quick question! How can I ensemble two models one of them trained with CPU and the other trained with GPU on Yolov5? Is it possible!!

[github-actions[bot]]

👋 Hello @AmmarAkran, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@AmmarAkran yes, see Model Ensembling tutorial:

YOLOv5 Tutorials

• Train Custom Data  🚀 RECOMMENDED
• Tips for Best Training Results  ☘️ RECOMMENDED
• Weights & Biases Logging  🌟 NEW
• Supervisely Ecosystem  🌟 NEW
• Multi-GPU Training
• PyTorch Hub  ⭐ NEW
• TorchScript, ONNX, CoreML Export 🚀
• Test-Time Augmentation (TTA)
• Model Ensembling
• Model Pruning/Sparsity
• Hyperparameter Evolution
• Transfer Learning with Frozen Layers  ⭐ NEW
• TensorRT Deployment

[AmmarOkran]

@glenn-jocher, I have followed the instruction and execute:

!python val.py --weights 'yolov5x_608.pt' 'best_2018_5x6.pt' --data rdd2018.yaml --img 608 --augment

I got this error:

val: data=/content/yolov5/data/rdd2018.yaml, weights=['yolov5x_608.pt', 'best_2018_5x6.pt'], batch_size=32, imgsz=608, conf_thres=0.001, iou_thres=0.6, task=val, device=, single_cls=False, augment=True, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-4-gb754525 torch 1.9.0+cu111 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)

Fusing layers... 
Model Summary: 476 layers, 87245797 parameters, 0 gradients
Fusing layers... 
Model Summary: 606 layers, 141063436 parameters, 0 gradients
Ensemble created with ['yolov5x_608.pt', 'best_2018_5x6.pt']

WARNING: --img-size 608 must be multiple of max stride 64, updating to 640
Traceback (most recent call last):
  File "val.py", line 360, in <module>
    main(opt)
  File "val.py", line 334, in main
    run(**vars(opt))
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "val.py", line 149, in run
    model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters())))  # run once
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/yolov5/models/experimental.py", line 81, in forward
    y.append(module(x, augment, profile, visualize)[0])
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/yolov5/models/yolo.py", line 126, in forward
    return self._forward_once(x, profile, visualize)  # single-scale inference, train
  File "/content/yolov5/models/yolo.py", line 149, in _forward_once
    x = m(x)  # run
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/yolov5/models/yolo.py", line 66, in forward
    y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

One model trained with CPU and the other with GPU!!!

[glenn-jocher]

@AmmarAkran 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. Your error is not reproducible:

We've created a few short guidelines below to help users provide what we need in order to get started investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

• ✅ Minimal – Use as little code as possible that still produces the same problem
• ✅ Complete – Provide all parts someone else needs to reproduce your problem in the question itself
• ✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

In addition to the above requirements, for Ultralytics to provide assistance your code should be:

• ✅ Current – Verify that your code is up-to-date with current GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been resolved by previous commits.
• ✅ Unmodified – Your problem must be reproducible without any modifications to the codebase in this repository. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all of the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template and providing a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

[AmmarOkran]

@glenn-jocher, I didn't get you, Do you mean I have to open a new issue with a bug title!!!

I wanted just to know if it is possible to ensemble two different devices trained models or not!

I am sorry for my bad English language!

[glenn-jocher]

@AmmarAkran yes, official models are the same as custom trained models, they are identical workflows. This is very easy to verify yourself:

# Train two YOLOv5s models on COCO128 for 3 epochs
!python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache
!python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache

# Ensemble Augmented Inference with 2 trained models
!python val.py --weights runs/train/exp/weights/best.pt runs/train/exp2/weights/best.pt --data coco128.yaml --img 640 --iou 0.65 --half --augment