how do I integrate my custom Yolov5 model with flask

[burhanuddin03]

❔Question

I want to integrate yolov5 model with flask and I am stuck. How do I convert best.pt to *.weights file? so after that I will convert it to tensorflow model and integrate it with flask.
Any other suggestions would be appreciable.

Additional context

[github-actions[bot]]

👋 Hello @burhanuddin03, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab Notebook with free GPU:
• Kaggle Notebook with free GPU: https://www.kaggle.com/ultralytics/yolov5
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Docker Image https://hub.docker.com/r/ultralytics/yolov5. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@burhanuddin03 YOLOv5 does not support export to the older darknet *.weights format.

Tensorflow export works well though, see #1127 for details.

[burhanuddin03]

What if I use YOLOv3 then it will possible?

[burhanuddin03]

@glenn-jocher

[Ownmarc]

why do you want to convert to Tensorflow ? Flask can work with Pytorch just fine! I serve all my yolov5 models with Flask without any issue.

[burhanuddin03]

@Ownmarc Can you please guide me how can I run detect.py file from flask or integrate it with flask api?

[decent-engineer-decent-datascientist]

I suggest you take a look at the torch hub tutorial (inference in a few lines that you could build into your project), or just read detect.py.

@glenn-jocher would you ever consider building a simple REST API to bundle into this project? With something like FastAPI, it'd only take a couple dozen lines of code.

[decent-engineer-decent-datascientist]

Thinking more than 20 seconds about it, never mind haha. It'd unnecessarily complicate the repo.

[glenn-jocher]

@decent-engineer-decent-datascientist I haven't included a model serving tutorial simply because it's not my specialty so I'm not qualified to create one. I know I've seen a lot of Flask and REST API questions though so there is significant community interest in this area. If minimal (or no) modifications are required to the repo and you know how to handle these then they might be very useful additions to the existing tutorials in https://docs.ultralytics.com/yolov5, and I could help with the associated PRs.

[glenn-jocher]

@decent-engineer-decent-datascientist the main avenue I know about is Torchserve https://pytorch.org/serve/, which seems to have some sort of AWS component available, though I admit I've never looked into it myself, so a tutorial would even benefit me at this point.

[decent-engineer-decent-datascientist]

Yea I can look into putting something together next week. I like the idea of using FastAPI as it does provide a little bit more customization, both in development and deployment. It also automagically generates swagger documentation which is dope.

Here's a simple demo of what it could look like: http://dsbyprateekg.blogspot.com/2020/06/how-to-deploy-your-ml-model-as-fast-api.html

[glenn-jocher]

@decent-engineer-decent-datascientist oh interesting! BTW, YOLOv5 hub inference also runs directly on web URIs, so if the content that needs inference is already hosted online (rare use case I admit), then all the model needs is the web URI of the file:

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

# Image
img = 'https://github.com/ultralytics/yolov5/raw/master/data/images/zidane.jpg'

# Inference
results = model(img)

[decent-engineer-decent-datascientist]

Nice! I got a bit of free time and threw together an example that uses just the web url functionality: https://gist.github.com/decent-engineer-decent-datascientist/81e04ad86e102eb083416e28150aa2a1

Just need to add the ability to receive files, and I think we'd be good to start integrating into detect.py

[decent-engineer-decent-datascientist]

You'll need to install these:

pip install fastapi
pip install uvicorn[standard]

[decent-engineer-decent-datascientist]

@burhanuddin03 does this gist also give you direction on how to use yolov5 with flask? The patterns are nearly the same.

[glenn-jocher]

@decent-engineer-decent-datascientist oh! That gist doesn't look too scary. It seems fastapi is doing most of the magic behind the scenes? How would one use the API for sending a local image?

Do you think we could eventually make a page with drag and drop upload capability for realtime YOLOv5 previews (assuming we kept a server up 24/7 to run this?

[decent-engineer-decent-datascientist]

@glenn-jocher Yessir, FastAPI will handle pretty much everything except the actual logic. Uploading files isn't bad either, take a look at https://fastapi.tiangolo.com/tutorial/request-files/. (this isn't in the gist)

Yea that wouldn't be terribly hard. An even moe intriguing proposition would be an ONNX.js deployment. That'd allow us to setup a page that hosts the inference code (drag/drop, preview results), but the processing would be done client side. After that we wouldn't even need a server, you could just host it off GitHub io.

This would make such a cool landing page for this project. That being said, this would be a semi-significant undertaking (10-20 hours minimum).

[decent-engineer-decent-datascientist]

Looks like there's even a yolo (granted v2) example.

[glenn-jocher]

@decent-engineer-decent-datascientist interesting, yes I think a smaller model like v5s would make a good js deployment option and a good demo for people looking to go down the web app route.

There's a route for a tensorflow js deployment also, using PR https://github.com/ultralytics/yolov5/pull/1127 for the Pytorch to TF conversion. I'm not sure exactly how the NMS would be handled in js deployments in general though, do you know?

[decent-engineer-decent-datascientist]

ONNX would be more performant, but tensorflow.js is probably more comfortable for most. tf.js has a nms function.

ONNX doesn't have one, but you could use something like this.

[glenn-jocher]

@decent-engineer-decent-datascientist oh awesome, so TF does have a js NMS module. Yes, the js deployments are very interesting, they can bypass a lot of compatibility issues at the expense of performance, for a single image processing they could provide a lot of value.

[WelkinU]

@decent-engineer-decent-datascientist oh! That gist doesn't look too scary. It seems fastapi is doing most of the magic behind the scenes? How would one use the API for sending a local image?

Do you think we could eventually make a page with drag and drop upload capability for realtime YOLOv5 previews (assuming we kept a server up 24/7 to run this?

I took some of the ideas from above and built a FastAPI wrapper using FileUpload instead of web URI. Also added some html forms/templates so you can upload an image and select model through a web browser. Code is here: https://github.com/WelkinU/yolov5-fastapi-demo

Here is an example of what it looks like in a web browser:

[glenn-jocher]

@WelkinU wow, very cool!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[theoll]

@WelkinU Is it possible to serve video instead image in your project?

[shubham12tomar]

❔Question

I want to integrate yolov5 model with flask and I am stuck. How do I convert best.pt to *.weights file? so after that I will convert it to tensorflow model and integrate it with flask. Any other suggestions would be appreciable.

Additional context

yes you can do it by using model = custom(path/to/best.pt) please refer yolov5/hubconf.py file