ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
49.37k stars 16.05k forks source link

How to get the coordinates of the bounding box in YOLO object detection? #388

Closed milind-soni closed 4 years ago

milind-soni commented 4 years ago

❔Question

I need to get the bounding box coordinates generated in an image using the object detection. How do I achieve that

github-actions[bot] commented 4 years ago

Hello @milind-soni, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook Open In Colab, Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note that Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

For more information please visit https://www.ultralytics.com.

glenn-jocher commented 4 years ago

@milind-soni detection results are available here: https://github.com/ultralytics/yolov5/blob/ea34f848a6afbe1fc0010745fdc5f356ed871909/detect.py#L92-L102

ankamdeepika commented 4 years ago

@glenn-jocher could you help me crop the detected objects using the bounding box coordinates

fahimnawaz7 commented 3 years ago

I too have the same issue

glenn-jocher commented 3 years ago

@fahimnawaz7 you can get the coordinates within detect.py, or you can use YOLOv5 from PyTorch Hub. See https://pytorch.org/hub/ultralytics_yolov5/

import cv2
import torch
from PIL import Image

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True).autoshape()  # for PIL/cv2/np inputs and NMS

# Images
for f in ['zidane.jpg', 'bus.jpg']:  # download 2 images
    print(f'Downloading {f}...')
    torch.hub.download_url_to_file('https://github.com/ultralytics/yolov5/releases/download/v1.0/' + f, f)
img1 = Image.open('zidane.jpg')  # PIL image
img2 = cv2.imread('bus.jpg')[:, :, ::-1]  # OpenCV image (BGR to RGB)
imgs = [img1, img2]  # batched list of images

# Inference
results = model(imgs, size=640)  # includes NMS

# Results
results.print()  # print results to screen
results.show()  # display results
results.save()  # save as results1.jpg, results2.jpg... etc.

# Data
print('\n', results.xyxy[0])  # print img1 predictions
#          x1 (pixels)  y1 (pixels)  x2 (pixels)  y2 (pixels)   confidence        class
# tensor([[7.47613e+02, 4.01168e+01, 1.14978e+03, 7.12016e+02, 8.71210e-01, 0.00000e+00],
#         [1.17464e+02, 1.96875e+02, 1.00145e+03, 7.11802e+02, 8.08795e-01, 0.00000e+00],
#         [4.23969e+02, 4.30401e+02, 5.16833e+02, 7.20000e+02, 7.77376e-01, 2.70000e+01],
#         [9.81310e+02, 3.10712e+02, 1.03111e+03, 4.19273e+02, 2.86850e-01, 2.70000e+01]])
AmelNozieres commented 3 years ago

Hello, is it good practice to modify the standard detect.py? what about if there is a new version? could we upgrade our code without any damage. How about we call detect.py with --save.txt, we parse the bounding boxes and we crop the image after that? Also is there a function as torch.hub.load to call with my weights?

glenn-jocher commented 3 years ago

@AmelNozieres yes YOLOv5 Hub models can be created from custom weights. See PyTorch Hub Tutorial for details:

Tutorials

fcakyon commented 3 years ago

@glenn-jocher is It possible to load custom weights in an environment that doesnt have internet access via yolov5 hub workflow? All examples I have come across require online access to the related github repo.

glenn-jocher commented 3 years ago

@fcakyon yes, if the hub cache and models are stored locally stored internet access is not needed.

glenn-jocher commented 3 years ago

@mycuriosity123 see torch hub page: https://pytorch.org/hub/ultralytics_yolov5/

glenn-jocher commented 3 years ago

@mycuriosity123 its assumed that users have at least a working knowledge of python here. To produce bounding box coordinates you simply copy and paste the code at the link I provided you:

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

# Images
dir = 'https://github.com/ultralytics/yolov5/raw/master/data/images/'
imgs = [dir + f for f in ('zidane.jpg', 'bus.jpg')]  # batched list of images

# Inference
results = model(imgs)

# Results
results.print()  
results.save()  # or .show()

# Data
print(results.xyxy[0])  # print img1 predictions (pixels)
#                   x1           y1           x2           y2   confidence        class
# tensor([[7.50637e+02, 4.37279e+01, 1.15887e+03, 7.08682e+02, 8.18137e-01, 0.00000e+00],
#         [9.33597e+01, 2.07387e+02, 1.04737e+03, 7.10224e+02, 5.78011e-01, 0.00000e+00],
#         [4.24503e+02, 4.29092e+02, 5.16300e+02, 7.16425e+02, 5.68713e-01, 2.70000e+01]])
glenn-jocher commented 3 years ago

@mycuriosity123 all the question you are asking are explained in the torch hub tutorial. I strongly recommend you start there.

Tutorials

mycuriosity123 commented 3 years ago

❔Question

I need to get the bounding box coordinates generated in an image using the object detection. How do I achieve that

have you achieved bounding box co ordinates?

mycuriosity123 commented 3 years ago

@glenn-jocher could you help me crop the detected objects using the bounding box coordinates

are you able to crop the detected objects?

AmelNozieres commented 3 years ago

@mycuriosity123, I don't know if this is what your looking for but if you need the bounding boxes generated by yolov5 you have to add --save-txt to your command !python detect.py --weights runs/train/yolov5s_results/weights/best.pt --img 416 --conf 0.4 --source ../test/images --save-txt after the detection is achieved you will see that for each image you have now a txt file with the bounding boxes. If you need to crop the image with this bounding boxes you can add some lines in detect.py. There is another issue with the code shared here https://github.com/ultralytics/yolov5/issues/803

mycuriosity123 commented 3 years ago

@mycuriosity123, I don't know if this is what your looking for but if you need the bounding boxes generated by yolov5 you have to add --save-txt to your command !python detect.py --weights runs/train/yolov5s_results/weights/best.pt --img 416 --conf 0.4 --source ../test/images --save-txt after the detection is achieved you will see that for each image you have now a txt file with the bounding boxes. If you need to crop the image with this bounding boxes you can add some lines in detect.py. There is another issue with the code shared here #803

thanks @AmelNozieres
hurray working!!!

SWRIGH211 commented 3 years ago

I've managed to distignuish the frames that have labelled bounding boxes in them by saving the txt bounding box coords found and put them into txt files, which then gives me an insight into what frame has labels and which ones don't since the txt files are saved as 00321.txt therefore I know frame 321 has a bounding box. is there a way to only call images that only have associated txt files?

AdityaKaran109 commented 3 years ago

@glenn-jocher could you help me crop the detected objects using the bounding box coordinates

                if save_img or view_img:  # Add bbox to image
                    label = '%s %.2f' % (names[int(cls)], conf)

####################################>>>>>>>>>> START of modified code <<<<<<<<<<<########################## #############>> commented this line to not to print bounding box on image plot_one_box(xyxy, im0, label=label, color=colors[int(cls)], line_thickness=3)

                    x1 = int(xyxy[0].item())
                    y1 = int(xyxy[1].item())
                    x2 = int(xyxy[2].item())
                    y2 = int(xyxy[3].item())
                    bbox_points=[x1, y1, x2, y2]
                    confidence_score = conf
                    class_index = cls
                    object_name = names[int(cls)]

                    print('bounding box is ', x1, y1, x2, y2)
                    print('class index is ', class_index)
                    print('detected object name is ', object_name)
                    original_img = im0

                    i=0
                    for i in bbox_points:
                        cropped_img = im0[y1:y2, x1:x2]
                        cv2.imwrite("/content/drive/MyDrive/YOLOV5_CUSTOM/crop/thumbnail%04i.png" %i, cropped_img) ###-----put the output folder path here---####
                        i+=1

                    #cv2.imwrite('test.png',cropped_img) ### >>>>>> to retain all cropped picture give different name for each pictures, else it will overwrite and only last image will be saved.

####################################>>>>>>>>>> END of modified code <<<<<<<<<<<##########################

Guemann-ui commented 3 years ago

Is it possible to get the pixel's values and coordinates under the bounding boxes (I'm using yolov5 PyTorch)? Thanks.

glenn-jocher commented 3 years ago

@besmaGuesmi you can modify detect.py prediction labels on L158: https://github.com/ultralytics/yolov5/blob/1f31b7c503867b6e8f493cf76ed2da490f834fd4/detect.py#L156-L162

Guemann-ui commented 3 years ago

I'll try it and back to you! thanks @glenn-jocher

glenn-jocher commented 3 years ago

@besmaGuesmi 👋 Hello! Thanks for asking about improving YOLOv5 🚀 training results.

Most of the time good results can be obtained with no changes to the models or training settings, provided your dataset is sufficiently large and well labelled. If at first you don't get good results, there are steps you might be able to take to improve, but we always recommend users first train with all default settings before considering any changes. This helps establish a performance baseline and spot areas for improvement.

If you have questions about your training results we recommend you provide the maximum amount of information possible if you expect a helpful response, including results plots (train losses, val losses, P, R, mAP), PR curve, confusion matrix, training mosaics, test results and dataset statistics images such as labels.png. All of these are located in your project/name directory, typically yolov5/runs/train/exp.

We've put together a full guide for users looking to get the best results on their YOLOv5 trainings below.

Dataset

COCO Analysis

Model Selection

Larger models like YOLOv5x and YOLOv5x6 will produce better results in nearly all cases, but have more parameters, require more CUDA memory to train, and are slower to run. For mobile deployments we recommend YOLOv5s/m, for cloud deployments we recommend YOLOv5l/x. See our README table for a full comparison of all models.

YOLOv5 Models

Training Settings

Before modifying anything, first train with default settings to establish a performance baseline. A full list of train.py settings can be found in the train.py argparser.

Further Reading

If you'd like to know more a good place to start is Karpathy's 'Recipe for Training Neural Networks', which has great ideas for training that apply broadly across all ML domains: http://karpathy.github.io/2019/04/25/recipe/

Nishant123412 commented 2 years ago

@milind-soni detection results are available here: https://github.com/ultralytics/yolov5/blob/ea34f848a6afbe1fc0010745fdc5f356ed871909/detect.py#L92-L102

plot_one_box ain't working anymore.

glenn-jocher commented 2 years ago

@Nishant123412 image annotations are now done using the new Annotator() class. See PR https://github.com/ultralytics/yolov5/pull/4591 for details.

baha2046a commented 2 years ago

I found the easiest way

import torch
model = torch.hub.load('ultralytics/yolov5', 'yolov5l6', pretrained=True)

frame = cv2.imread(path)
detections = model(frame[..., ::-1])
results = detections.pandas().xyxy[0].to_dict(orient="records")
for result in results:
                con = result['confidence']
                cs = result['class']
                x1 = int(result['xmin'])
                y1 = int(result['ymin'])
                x2 = int(result['xmax'])
                y2 = int(result['ymax'])
                # Do whatever you want
                cv2.rectangle(frame, (x1, y1), (x2, y2), COLORS[0], 2)
debjyoti003 commented 2 years ago

@glenn-jocher what I found is number of objects found using detect.py and torch.hub.load are different. Can you please tell me why is that? Am I doing something wrong or is it something else?

glenn-jocher commented 2 years ago

@debjyoti003 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. We've created a few short guidelines below to help users provide what we need in order to get started investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

For Ultralytics to provide assistance your code should also be:

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

mansi733 commented 2 years ago

I found the easiest way

import torch
model = torch.hub.load('ultralytics/yolov5', 'yolov5l6', pretrained=True)

frame = cv2.imread(path)
detections = model(frame[..., ::-1])
results = detections.pandas().xyxy[0].to_dict(orient="records")
for result in results:
                con = result['confidence']
                cs = result['class']
                x1 = int(result['xmin'])
                y1 = int(result['ymin'])
                x2 = int(result['xmax'])
                y2 = int(result['ymax'])
                # Do whatever you want
                cv2.rectangle(frame, (x1, y1), (x2, y2), COLORS[0], 2)

Can we hide labels and confidence scores using this code, I just want to plot the bounding box while doing GUI inference?

glenn-jocher commented 2 years ago

@mansi733 @baha2046a good news 😃! Your original issue may now be fixed ✅ in PR #7129. This PR allows for showing/saving results without labels (boxes only).

Usage:

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # or yolov5m, yolov5l, yolov5x, custom

# Images
img = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, PIL, OpenCV, numpy, list

# Inference
results = model(img)

# Results
results.render(labels=False)
results.show(labels=False)
results.save(labels=False)

To receive this update:

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

mansi733 commented 2 years ago

@mansi733 @baha2046a good news 😃! Your original issue may now be fixed ✅ in PR #7129. This PR allows for showing/saving results without labels (boxes only).

Usage:

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # or yolov5m, yolov5l, yolov5x, custom

# Images
img = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, PIL, OpenCV, numpy, list

# Inference
results = model(img)

# Results
results.render(labels=False)
results.show(labels=False)
results.save(labels=False)

To receive this update:

  • Gitgit pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
  • PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
  • Notebooks – View updated notebooks Open In Colab Open In Kaggle
  • Dockersudo docker pull ultralytics/yolov5:latest to update your image Docker Pulls

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

Thanks!! now I am getting this error -->TypeError: render() got an unexpected keyword argument 'labels'. can you please look into this?

glenn-jocher commented 2 years ago

@mansi733 you need to update your code as my previous message indicates.

To receive this update:

  • Gitgit pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
  • PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
  • Notebooks – View updated notebooks Open In Colab Open In Kaggle
  • Dockersudo docker pull ultralytics/yolov5:latest to update your image Docker Pulls
glenn-jocher commented 2 years ago
Screenshot 2022-03-24 at 13 54 18
Akotkar23 commented 2 years ago

Can we hide confidence score in a same way using render. Tried out but giving error as unexpected keyword argument. Error: tempimg=pred.render(confidence=False) TypeError: render() got an unexpected keyword argument 'confidence'

DrkPlne commented 2 years ago

@milind-sonialgılama sonuçları burada mevcuttur: https://github.com/ultralytics/yolov5/blob/ea34f848a6afbe1fc0010745fdc5f356ed871909/detect.py#L92-L102

plot_one_box artık çalışmıyor.

Hey, Hello there Did you find a solution to this problem?

glenn-jocher commented 2 years ago

@DrkPlne 👋 Hello! Thanks for asking about handling inference results. YOLOv5 🚀 PyTorch Hub models allow for simple model loading and inference in a pure python environment without using detect.py.

Simple Inference Example

This example loads a pretrained YOLOv5s model from PyTorch Hub as model and passes an image for inference. 'yolov5s' is the YOLOv5 'small' model. For details on all available models please see the README. Custom models can also be loaded, including custom trained PyTorch models and their exported variants, i.e. ONNX, TensorRT, TensorFlow, OpenVINO YOLOv5 models.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # yolov5n - yolov5x6 official model
#                                            'custom', 'path/to/best.pt')  # custom model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.

results.xyxy[0]  # im predictions (tensor)
results.pandas().xyxy[0]  # im predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

See YOLOv5 PyTorch Hub Tutorial for details.

Good luck 🍀 and let us know if you have any other questions!

muelclamor commented 2 years ago

@mycuriosity123 its assumed that users have at least a working knowledge of python here. To produce bounding box coordinates you simply copy and paste the code at the link I provided you:

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

# Images
dir = 'https://github.com/ultralytics/yolov5/raw/master/data/images/'
imgs = [dir + f for f in ('zidane.jpg', 'bus.jpg')]  # batched list of images

# Inference
results = model(imgs)

# Results
results.print()  
results.save()  # or .show()

# Data
print(results.xyxy[0])  # print img1 predictions (pixels)
#                   x1           y1           x2           y2   confidence        class
# tensor([[7.50637e+02, 4.37279e+01, 1.15887e+03, 7.08682e+02, 8.18137e-01, 0.00000e+00],
#         [9.33597e+01, 2.07387e+02, 1.04737e+03, 7.10224e+02, 5.78011e-01, 0.00000e+00],
#         [4.24503e+02, 4.29092e+02, 5.16300e+02, 7.16425e+02, 5.68713e-01, 2.70000e+01]])

How can i adjust the bounding box thickness in this code ? newbie in using yolov5 thanks for the reply in advance

glenn-jocher commented 2 years ago

@muelclamor you might modify the source code or annotator arguments here and point pytorch hub to a local YOLOv5 clone:

https://github.com/ultralytics/yolov5/blob/29d79a6360d8c7da8875284246847db3312e270a/models/common.py#L651-L664

muelclamor commented 2 years ago

@muelclamor you might modify the source code or annotator arguments here and point pytorch hub to a local YOLOv5 clone:

https://github.com/ultralytics/yolov5/blob/29d79a6360d8c7da8875284246847db3312e270a/models/common.py#L651-L664

@glenn-jocher thank you it work and i have another question, is it possible to assign a variable for a certain bounding box or is it not ? i want to assign a variable when the yolov5 detect a certain object then connect it to my excel to output it via arduino thanks in advance

glenn-jocher commented 2 years ago

@muelclamor you can export results.pandas() to Excel, see https://pandas.pydata.org/docs/reference/api/pandas.ExcelWriter.html

rodrigoviannini commented 2 years ago

Olá, é uma boa prática modificar o detect.py padrão? e se houver uma nova versão? poderíamos atualizar nosso código sem nenhum dano. Que tal chamarmos detect.py com --save.txt, analisamos as caixas delimitadoras e cortamos a imagem depois disso? Também existe uma função como torch.hub.load para chamar com meus pesos?

Gostaria de saber tambem!

glenn-jocher commented 2 years ago

@rodrigoviannini 👋 Hello! Thanks for asking about handling inference results. YOLOv5 🚀 PyTorch Hub models allow for simple model loading and inference in a pure python environment without using detect.py.

Simple Inference Example

This example loads a pretrained YOLOv5s model from PyTorch Hub as model and passes an image for inference. 'yolov5s' is the YOLOv5 'small' model. For details on all available models please see the README. Custom models can also be loaded, including custom trained PyTorch models and their exported variants, i.e. ONNX, TensorRT, TensorFlow, OpenVINO YOLOv5 models.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # yolov5n - yolov5x6 official model
#                                            'custom', 'path/to/best.pt')  # custom model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.
results.xyxy[0]  # im predictions (tensor)

results.pandas().xyxy[0]  # im predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

results.pandas().xyxy[0].value_counts('name')  # class counts (pandas)
# person    2
# tie       1

See YOLOv5 PyTorch Hub Tutorial for details.

Good luck 🍀 and let us know if you have any other questions!

OttomanZ commented 1 year ago

Simple Example for Custom Model


import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # or yolov5m, yolov5l, yolov5x, etc.
# model = torch.hub.load('ultralytics/yolov5', 'custom', 'path/to/best.pt')  # custom trained model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.

results.xyxy[0]  # im predictions (tensor)
results.pandas().xyxy[0]  # im predictions (pandas)
daquarti commented 1 year ago

simple you can use "--save-txt" and or "--save-crop" arguments in python detect.py

Example in colab !python detect.py --weights /content/yolov5/runs/train/exp/weights/best.pt --img 640 --conf 0.01 --source data/images/ --save-txt --save-crop

CyberpunkCoder101 commented 1 year ago

@milind-soni detection results are available here:

https://github.com/ultralytics/yolov5/blob/ea34f848a6afbe1fc0010745fdc5f356ed871909/detect.py#L92-L102

which variable specifically hold values of co ordinates of bounding boxes?

arshanabbas commented 1 year ago

I have few questions. I am trying to measure the height and width of an object in YoloV5. The road map I am having in my mind is that the coordinates of bounding box are available and can be saved with --save-txt command, so with these bounding box coordinates we can calculate Pixel in selected area with OpenCV and as per the size of the image we can calculate height and width although better way is to use Aruco marker but I am leaving the Aruco marker step for now.

lienpham83 commented 1 year ago

Hi glenn-jocher, I would like get the bounding box with the absolute coordinates from YOLOv5 predictions. Could you please help me to do this? In the Simple Example for Custom Model, I can get only the relative coordinates. Many thanks

swn0000 commented 1 year ago

too hard to me

ChanHaoHao commented 1 year ago

Maybe it will be easier to use --save-txt to get the coordinates of the boxes of each figure in YOLO form.

mathhauk commented 1 year ago

Is it possible to get coordinates for the bounding boxes when the object detection is used on a video? Can't seem to find this anywhere.

glenn-jocher commented 9 months ago

@mathhauk 👋 Hello! Yes, it's absolutely possible to obtain bounding box coordinates from YOLOv5 object detection in a video. In fact, this can be easily achieved using YOLOv5 by saving detection results to a text file using the --save-txt argument in the inference script. Each text file will contain the bounding box coordinates of the detected objects in YOLO format (x_center, y_center, width, height).

Afterwards, these coordinates can be utilized to calculate pixel locations using OpenCV or to convert to absolute coordinates based on the size of the image or frame from the video. If you have any other questions or need further assistance, feel free to ask!