YOLOv5 OBB Inference with OpenCV DNN

zijian98 commented 1 year ago

Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi, I have tested out one of the variant of yolov5 which can detect oriented bounding boxes in DOTA dataset and I am wondering if anyone had successfully implemented their model and run inference with C++ code?

This is the repo but the author seems to be inactive: https://github.com/hukaixuan19970627/yolov5_obb

Thank You!

Additional

No response

github-actions[bot] commented 1 year ago

👋 Hello @zijian98, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

glenn-jocher commented 1 year ago

@zijian98 hi there,

Thank you for reaching out. While I'm not aware of anyone specifically implementing the YOLOv5-oriented bounding box (OBB) variant and running inference with C++ code, it is certainly possible to do so.

You can start by referring to the YOLOv5 PyTorch Hub for the official PyTorch model weights. From there, you can use a PyTorch C++ extension to load the model and run inference on your images.

To implement the YOLOv5 OBB variant, you can take inspiration from the repository you shared (https://github.com/hukaixuan19970627/yolov5_obb). While the author may be inactive, you can still examine the codebase and adapt it to your needs.

Feel free to ask for further clarification or assistance if needed. Good luck with your project!

zijian98 commented 1 year ago

Hi @glenn-jocher ,

Thanks for the reference links. Maybe I can start off with running inference with an exported ONNX model using Python OpenCV DNN first. I have read through the codes for both your YOLOv5 and the obb variant and the main difference would be the input data having an extra "theta" parameter to determine the angle of rotation of the bounding box.

The issue I am facing is extracting the detected bbox information from the output. From your pre-trained YOLOv5s, I understand that the output shape (25200, 85) represents 25200 rows of predictions with 85 values in it (5 for bbox information and confidence and 80 for class probabilities).

Using the same preprocessing method for the OBB variant also allows me the generate the correct output shape dimension (25200 201) which can be verified in their Detect.py script. However, I am not sure why are there 201 values in each row when we only trained 16 classes from DOTA dataset.

This may not be the right repo to ask but hope that you can share some expertise regarding this post processing. You can also correct me should I mentioned any wrong information based on my own understanding.

Thank You!

Reference python code for usage with original YOLOv5 models: https://github.com/doleron/yolov5-opencv-cpp-python/blob/main/python/yolo.py

github-actions[bot] commented 1 year ago

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

Docs: https://docs.ultralytics.com
HUB: https://hub.ultralytics.com
Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

glenn-jocher commented 11 months ago

@zijian98,

You're correct that the OBB variant will have an additional "theta" parameter for the bounding box rotation angle. As for the output shape from the model, while the standard YOLOv5 model outputs a tensor of shape (N, 85), the OBB variant should output a tensor of shape (N, 201) for 16 classes.

This discrepancy could be due to the format used for representing the oriented bounding boxes, which may include additional parameters such as the rotation angle and dimensions. To better understand the output format, I recommend consulting the specific implementation details within the OBB variant, especially the Detect.py script you mentioned.

Although I can't provide specific details on the OBB variant repository you referenced, you can consider examining the post-processing steps within their code to understand how the output tensor is handled and interpreted for inference results.

Remember, the YOLOv5 community and the Ultralytics team are valuable resources for expertise on model variants and their implementations. Best of luck, and feel free to ask for further clarification or assistance.

Sriyab002 commented 9 months ago

@glenn-jocher hello. i have been working on point cloud object detection and was wondering if i can use yolov5_obb model. I could train the model with 2D bird eye view images of the kitti dataset and implement detection right?

ultralytics / yolov5