Closed iann838 closed 2 years ago
👋 Hello @paaksing, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.
If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.
If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.
For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.
Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:
git clone https://github.com/ultralytics/yolov5 # clone
cd yolov5
pip install -r requirements.txt # install
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
This indirectly affects compilation of artifacts to aws neuron runtime (torch.neuron.trace
), cutting out access to critical low cost high throughput hardware that supports many infrastructures.
@paaksing 👋 Hello! Thanks for asking about Export Formats. YOLOv5 🚀 offers export to most popular formats used today. See our TFLite, ONNX, CoreML, TensorRT Export Tutorial for details.
YOLOv5 inference is officially supported in 11 formats:
💡 ProTip: TensorRT may be up to 2-5X faster than PyTorch on GPU benchmarks 💡 ProTip: ONNX and OpenVINO may be up to 2-3X faster than PyTorch on CPU benchmarks
Format | export.py --include |
Model |
---|---|---|
PyTorch | - | yolov5s.pt |
TorchScript | torchscript |
yolov5s.torchscript |
ONNX | onnx |
yolov5s.onnx |
OpenVINO | openvino |
yolov5s_openvino_model/ |
TensorRT | engine |
yolov5s.engine |
CoreML | coreml |
yolov5s.mlmodel |
TensorFlow SavedModel | saved_model |
yolov5s_saved_model/ |
TensorFlow GraphDef | pb |
yolov5s.pb |
TensorFlow Lite | tflite |
yolov5s.tflite |
TensorFlow Edge TPU | edgetpu |
yolov5s_edgetpu.tflite |
TensorFlow.js | tfjs |
yolov5s_web_model/ |
Full CPU benchmarks
benchmarks: weights=/content/yolov5/yolov5s.pt, imgsz=640, batch_size=1, data=/content/yolov5/data/coco128.yaml, device=cpu, half=False, test=False
Checking setup...
YOLOv5 🚀 v6.1-135-g7926afc torch 1.10.0+cu111 CPU
Setup complete ✅ (8 CPUs, 51.0 GB RAM, 41.5/166.8 GB disk)
Benchmarks complete (241.20s)
Format mAP@0.5:0.95 Inference time (ms)
0 PyTorch 0.4623 127.61
1 TorchScript 0.4623 131.23
2 ONNX 0.4623 69.34
3 OpenVINO 0.4623 66.52
4 TensorRT NaN NaN
5 CoreML NaN NaN
6 TensorFlow SavedModel 0.4623 123.79
7 TensorFlow GraphDef 0.4623 121.57
8 TensorFlow Lite 0.4623 316.61
9 TensorFlow Edge TPU NaN NaN
10 TensorFlow.js NaN NaN
Full GPU benchmarks
benchmarks: weights=/content/yolov5/yolov5s.pt, imgsz=640, batch_size=1, data=/content/yolov5/data/coco128.yaml, device=0, half=False, test=False
Checking setup...
YOLOv5 🚀 v6.1-135-g7926afc torch 1.10.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
Setup complete ✅ (8 CPUs, 51.0 GB RAM, 46.7/166.8 GB disk)
Benchmarks complete (458.07s)
Format mAP@0.5:0.95 Inference time (ms)
0 PyTorch 0.4623 10.19
1 TorchScript 0.4623 6.85
2 ONNX 0.4623 14.63
3 OpenVINO NaN NaN
4 TensorRT 0.4617 1.89
5 CoreML NaN NaN
6 TensorFlow SavedModel 0.4623 21.28
7 TensorFlow GraphDef 0.4623 21.22
8 TensorFlow Lite NaN NaN
9 TensorFlow Edge TPU NaN NaN
10 TensorFlow.js NaN NaN
Good luck 🍀 and let us know if you have any other questions!
@paaksing can you provide a reproducible successful command with v6.1 please? This should help us to understand the issue, as the above are not part of our CI or standard use cases.
@glenn-jocher
!pip install torch==1.8.1 torchvision==0.9.1
import torch
import requests
with open("yolov5s61.pt", "wb+") as f:
f.write(requests.get("https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt").content)
model = torch.hub.load('ultralytics/yolov5:v6.1', 'custom', path="yolov5s61.pt", force_reload=True)
try:
torch.jit.trace(model, [torch.zeros([1, 3, 640, 640])])
except Exception:
torch.jit.trace(model, [torch.zeros([1, 3, 640, 640])])
This works, and is very similar with torch.neuron.trace
(See #8619), I don't know what is the reason that the tracing has to be called twice 🤷, it already looks weird in v6.1 having to catch and repeat the call. Also another issue is that torch.hub.load
, by specifying model = torch.hub.load('ultralytics/yolov5:v6.1', "yolov5s.pt", force_reload=True)
it will download yolov5s
from v6.2 regarless of the repository tag/branch, so I needed to download the weights manually and use it as custom. This code uses torch 1.8.1 because 1.12 gives the error on #6948
@paaksing yes thanks! The Try Except appears redundant, perhaps it exists to catch model download issues.
In any case the v6.1 and v6.2 detection models are exactly identical files, same hash, same weights, they have not been retrained, only classification models have been added.
When we export torchscript models in export.py we also run torch.jit.trace and this export works correctly (tested every 24 hours in CI tests): https://github.com/ultralytics/yolov5/blob/4e8504abd9c1a7287dfcf9f96dfa04f061086cca/export.py#L112-L126
@glenn-jocher I tried to get as close as that script, but the same error keeps coming, maybe is the way the model is loaded using torch.hub.load
be any different ? I tried modifying inplace
and fuse
as well with no success. Let me try loading the models without hub.
@paaksing I tested your code with current torch and it fails:
Also installed older torch as in your example and also fails:
@glenn-jocher Looks fine for me
Also, it appears that the first trace call will always raise an exception, except the second one
Got it. When running two it works, this is probably just because the model needs a warmup to build grids. Anyway, v6.2 works correctly for me when I run it twice:
@paaksing ok I figured this out. v6.2 passes the test, but master does not, so a change between v6.2 and now has caused this. You can use git bisect to track this down by passing in the exact commit hash, i.e. here from July 30th this commit passes. Can you help test commits until you find the first that fails?
commit hash 1e89807d9a208727e3f0e9bf26a1e286d0ce416b https://github.com/ultralytics/yolov5/commit/1e89807d9a208727e3f0e9bf26a1e286d0ce416b
All commits at https://github.com/ultralytics/yolov5/commits/master
!pip install torch==1.8.1 torchvision==0.9.1
import torch
import requests
model = torch.hub.load('ultralytics/yolov5:1e89807d9a208727e3f0e9bf26a1e286d0ce416b', 'custom', path="https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt", force_reload=True, autoshape=False)
model.cpu()
try:
torch.jit.trace(model, [torch.zeros([1, 3, 640, 640])])
except Exception:
torch.jit.trace(model, [torch.zeros([1, 3, 640, 640])])
@glenn-jocher sure
@glenn-jocher found it, 7aa263c5f2f526472435babf86ddd33eed1dbd78
@paaksing oh that was fast. Yes that makes sense, that changed DetectMultiBackend behavior. Ok I'll to find a fix, and I'll also try to add this workflow to the CI to safeguard the workflow in the future.
@glenn-jocher One by one is slow, so I went the binary search method. Thanks, I'll wait for news
@paaksing ok https://github.com/ultralytics/yolov5/pull/9363 appears to be working. As long as you leave autoshape=True
(the default) on PyTorch Hub model load you should be fine to jit trace.
I'm going to add some CI and then merge.
@paaksing here's the test script that traces v6.2 models with latest torch and PR (cleaned up and using warmup rather than Try Except:
import torch
import requests
model = torch.hub.load('ultralytics/yolov5:update/torch', 'yolov5s', force_reload=True, skip_validation=True)
model.cpu()
im = torch.zeros([1, 3, 640, 640])
model(im) # warmup, build grids
torch.jit.trace(model, [im])
EDIT: note skip_validation=True
required with latest torch releases
@glenn-jocher Thanks a lot, It's working and now the script looks cleaner as well, closing this.
@paaksing good news 😃! Your original issue may now be fixed ✅ in PR #9363. This PR also adds torch.jit.trace() CI to protect from tracing issues arising in the future.
To receive this update:
git pull
from within your yolov5/
directory or git clone https://github.com/ultralytics/yolov5
againmodel = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
sudo docker pull ultralytics/yolov5:latest
to update your image Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!
Search before asking
YOLOv5 Component
Export
Bug
Environment
YOLOv5 🚀 2022-9-9 Python-3.7.13 torch-1.12.1+cu113 CUDA:0 (Tesla P100-PCIE-16GB, 16281MiB)
Minimal Reproducible Example
Additional
v6.1 works fine, but breaks after updating to v6.2
Are you willing to submit a PR?