running default yolov5 on jetson nano, but the fps is just under 1 fps

[HuumbleBee]

Helo, i have jetson nano 2gb, i try to run default yolov5 but the fps is just under 1 fps on it. what i should do to improve it? i also plan to use custom dataset in the future, can you give me some advice to gain best result when train dataset? thanks

[github-actions[bot]]

👋 Hello @HuumbleBee, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[AyushExel]

@HuumbleBee You need to export the model to tensorRT format for running inference on it. Here's the command in for converting yolov5s model to TRT: python export.py --weights yolov5s.pt --include engine You can then use the exported model for inference. Let me know if you have any questions.

[HuumbleBee]

Thanks, i will try it asap

[SkalskiP]

Hi, @AyushExel sorry to interrupt the conversation here, but:

1. Do you maybe have some benchmarks comparing YOLOv5 performance between Torch and TRT?
2. Do I still need to have torch and torch vision installed in my Python environment if I infer using TRT?

[glenn-jocher]

@SkalskiP yes GPU benchmarks on a few systems are available here: https://github.com/ultralytics/yolov5/pull/6963

TRT inference with YOLOv5 detect.py/val.py/hub uses torch but there is a torch-free export PR open here you might be able to use: https://github.com/ultralytics/yolov5/pull/7736

[SkalskiP]

Hi, @glenn-jocher thanks for the response! So it seems to me that we are getting a consistent 3-5x boost across all tested platforms. Shame tho... that there are no benchmarks for Jetson Xavier / Nano.

1. So as it is right now, the inference is done by TRT, but NMS and other post-processing are done using torch?
2. Does the pip package support TRT inference?

[glenn-jocher]

@SkalskiP no we don't have any edge device benchmarks other than the iOS/Android app, but this is just because we don't have any edge devices to test on nor anyone in place to manage/verify/benchmark edge devices, i.e. Edge TPU, NCS2, RPi, Jetson etc. For Xavier/Nano specifically I think you need to export and benchmark on the same device too.

Yes torch is mainly just used for NMS on TRT models. The pip package is not maintained adequately unfortunately so I think it may be a bit behind master in terms of features. I don't know if supports TRT, but we do have plans to launch a more official and maintained pip package soon.

[SkalskiP]

@glenn-jocher I could do some benchmarking on Jetson Xavier, I'd just need some instructions on how to do it.

[glenn-jocher]

@SkalskiP I've never done it myself, but I think that the jetson docker images are the best way to get started quickly: https://developer.nvidia.com/embedded/learn/tutorials/jetson-container

[SkalskiP]

@glenn-jocher aaa you misunderstood my question. 😊 I'm not asking about the Jetson Xavier instructions. I have that already figured out, I actually managed to set it up with python 3.8 which I guess used to be a problem? I remember that only 3.6 was supported.

I meant instructions for benchmarking. 😉

[glenn-jocher]

@SkalskiP oh the benchmarking is simple, just run python utils/benchmarks.py to benchmark all formats.

Try to install all dependencies before running. Some will auto install, others like TF will not.

[chunxiao365]

Helo, i have jetson nano 2gb, i try to run default yolov5 but the fps is just under 1 fps on it. what i should do to improve it? i also plan to use custom dataset in the future, can you give me some advice to gain best result when train dataset? thanks

which jetpack version do you installed on jetson nano ? I tried complied from source and docker run ,both failed .

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[glenn-jocher]

@chunxiao365 hi there! For Jetson Nano, optimizing YOLOv5 performance involves ensuring you have the correct JetPack version, using the latest YOLOv5 version, and exploring torch/tensorRT model export options for improved inference speeds. For custom dataset training, refer to the Ultralytics Docs for guidance, and ensure your dataset is diverse and well-labeled. Good luck!