Pytorch error in scale_box

[Maioy97]

Search before asking

• [X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I have code that uses yolov5 for detection, I use it on mutiple machines and it works on most of them this is the code that uses it

            # logger.info("in_cam -- face detection end")

            for i, det in enumerate(results.xyxy):
                logger.info("detections have been detected")
                print(det[:, :4])
                img1shape = frame.shape[0:2]
                detected_coordinates = det[:, :4]
                img0shape = frame.shape
                detected_coordinates = scale_boxes(img1shape, detected_coordinates, img0shape).round()
                xywhs = xyxy2xywh(det[:, 0:4])
                confs = detected_coordinates
                clss = detected_coordinates

on this machine it gives me this errors:

  File "/path_to_project/in_cam_code.py", line 721, in in_process
    detected_coordinates = scale_boxes(img1shape, detected_coordinates, img0shape).round()
  File "/path_to_project/yolov5/utils/general.py", line 781, in scale_boxes
    boxes[:, [0, 2]] -= pad[0]  # x padding
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

I think it might be an environment issue I did install the requirements.txt I tried updating the repo then reinstalling, still the same issue I'm not sure which library version might lead to this here's my pip list result just in case

(tfremake) mai@mai-ThinkPad-E15-Gen-2:~$ pip list 
Package                       Version
----------------------------- --------------------
absl-py                       1.2.0
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
asttokens                     2.0.8
astunparse                    1.6.3
attrs                         22.1.0
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4                4.11.1
bleach                        5.0.1
Bottleneck                    1.3.5
cachetools                    5.2.0
certifi                       2022.9.24
cffi                          1.15.1
charset-normalizer            2.1.1
click                         8.1.3
cycler                        0.11.0
debugpy                       1.6.3
decorator                     5.1.1
defusedxml                    0.7.1
dlib                          19.24.0
easydict                      1.9
entrypoints                   0.4
executing                     1.1.0
face-recognition              1.3.0
face-recognition-models       0.3.0
fastjsonschema                2.16.2
filelock                      3.8.0
fire                          0.4.0
Flask                         2.2.2
flatbuffers                   2.0.7
flit_core                     3.7.1
fonttools                     4.25.0
gast                          0.4.0
gdown                         4.5.1
google-auth                   2.11.1
google-auth-oauthlib          0.4.6
google-pasta                  0.2.0
grpcio                        1.49.1
h5py                          3.7.0
idna                          3.4
importlib-metadata            4.11.4
importlib-resources           5.9.0
ipykernel                     6.16.0
ipython                       8.5.0
ipython-genutils              0.2.0
itsdangerous                  2.1.2
jedi                          0.18.1
Jinja2                        3.1.2
joblib                        1.2.0
jsonschema                    4.16.0
jupyter_client                7.3.5
jupyter_core                  4.11.1
jupyterlab-pygments           0.2.2
keras                         2.8.0
Keras-Preprocessing           1.1.2
kiwisolver                    1.4.2
libclang                      14.0.6
lightgbm                      3.3.2
lxml                          4.9.1
Markdown                      3.4.1
MarkupSafe                    2.1.1
matplotlib                    3.5.2
matplotlib-inline             0.1.6
mediapipe                     0.8.11
mistune                       2.0.4
mtcnn                         0.1.1
munkres                       1.1.4
nb-conda                      2.2.1
nb-conda-kernels              2.3.1
nbclient                      0.6.8
nbconvert                     7.0.0
nbformat                      5.6.1
nest-asyncio                  1.5.5
notebook                      6.4.12
numexpr                       2.8.3
numpy                         1.21.5
oauthlib                      3.2.1
opencv-contrib-python         4.6.0.66
opencv-python                 4.6.0.66
opt-einsum                    3.3.0
packaging                     21.3
pandas                        1.4.4
pandocfilters                 1.5.0
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.2.0
pip                           22.2.2
pkgutil_resolve_name          1.3.10
ply                           3.11
prometheus-client             0.14.1
prompt-toolkit                3.0.31
protobuf                      3.19.5
psutil                        5.9.2
ptyprocess                    0.7.0
pure-eval                     0.2.2
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycparser                     2.21
Pygments                      2.13.0
pyparsing                     3.0.9
PyQt5-sip                     12.11.0
pyrsistent                    0.18.1
PySocks                       1.7.1
python-dateutil               2.8.2
pytz                          2022.2.1
PyYAML                        6.0
pyzmq                         24.0.1
requests                      2.28.1
requests-oauthlib             1.3.1
retina-face                   0.0.12
rsa                           4.9
scikit-learn                  1.1.2
scipy                         1.7.3
seaborn                       0.11.2
Send2Trash                    1.8.0
setuptools                    65.4.0
sip                           6.6.2
six                           1.16.0
soupsieve                     2.3.2.post1
stack-data                    0.5.1
tensorboard                   2.8.0
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow                    2.8.0
tensorflow-estimator          2.10.0
tensorflow-io-gcs-filesystem  0.27.0
termcolor                     2.0.1
terminado                     0.15.0
tf-estimator-nightly          2.8.0.dev2021122109
thop                          0.1.1.post2209072238
threadpoolctl                 3.1.0
tinycss2                      1.1.1
toml                          0.10.2
torch                         1.10.1+cpu
torchaudio                    0.10.1+rocm4.1
torchvision                   0.11.2+cpu
tornado                       6.2
tqdm                          4.64.0
traitlets                     5.4.0
typing_extensions             4.3.0
urllib3                       1.26.12
wcwidth                       0.2.5
webencodings                  0.5.1
Werkzeug                      2.2.2
wheel                         0.37.1
wrapt                         1.14.1
zipp                          3.8.1

Additional

No response

[github-actions[bot]]

👋 Hello @Maioy97, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@Maioy97 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. We've created a few short guidelines below to help users provide what we need in order to start investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

• ✅ Minimal – Use as little code as possible to produce the problem
• ✅ Complete – Provide all parts someone else needs to reproduce the problem
• ✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

For Ultralytics to provide assistance your code should also be:

• ✅ Current – Verify that your code is up-to-date with GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been solved in master.
• ✅ Unmodified – Your problem must be reproducible using official YOLOv5 code without changes. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

[Maioy97]

the issue keeps re emerging and fixing itself whenever I change surrounding files I'll document it once it comes back

[Maioy97]

Hello, I went back to the broken state and made a reproducible error

import sys
sys.path.insert(0, './yolov5')
from yolov5.utils.general import scale_boxes, xyxy2xywh
import torch
import cv2

def main(image, yolo_model):
    results = yolo_model(image, size=640)
    # logger.info("in_cam -- face detection end")
    for i, detect_info in enumerate(results.xyxy):
        print("detections have been detected")
        # print(det[:, :4])
        img1shape = image.shape[0:2]
        detected_coordinates = detect_info[:, :4]
        img0shape = image.shape
        detect_info[:, :4] = scale_boxes(img1shape, detected_coordinates, img0shape).round()
        xywhs = xyxy2xywh(detect_info[:, 0:4])
        confs = detect_info[:, 4]
        clss = detect_info[:, 5]
    print("do something with the reshaped results")

if __name__ == "__main__":
    image_path = './martha.jpg'
    image = cv2.imread(image_path)
    model = torch.hub.load('yolov5', 'custom', path='assets/models/best.pt', source='local')
    model.conf = 0.6
    model.iou = 0.1
    main(image, model)

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[Maioy97]

@glenn-jocher

[glenn-jocher]

@Maioy97 hi there. So you think there is an error in scale_boxes in master? Does the error occur in master code with normal usage or does it only occur when embedding in custom code like in your example https://github.com/ultralytics/yolov5/issues/9629#issuecomment-1282532933 ?

If you have a solution can you please submit a PR directly to help us evaluate it? Thanks!

[Maioy97]

I'm not sure what you meant by normal usage but I mostly get the errors when I pass the coordinates to the scale_boxes function the way I stated above

[glenn-jocher]

@Maioy97 got it. I tried to use your code to reproduce but ran into an unrelated error. I cloned YOLOv5, cd into it and run your code with yolov5s.pt and zidane.jpg (since I don't have your model and your image). Please make sure your code is actually runnable:

!git clone https://github.com/ultralytics/yolov5  # clone
%cd yolov5
%pip install -qr requirements.txt  # install

import sys
sys.path.insert(0, './yolov5')
from yolov5.utils.general import scale_boxes, xyxy2xywh
import torch
import cv2

def main(image, yolo_model):
    results = yolo_model(image, size=640)
    # logger.info("in_cam -- face detection end")
    for i, detect_info in enumerate(results.xyxy):
        print("detections have been detected")
        # print(det[:, :4])
        img1shape = image.shape[0:2]
        detected_coordinates = detect_info[:, :4]
        img0shape = image.shape
        detect_info[:, :4] = scale_boxes(img1shape, detected_coordinates, img0shape).round()
        xywhs = xyxy2xywh(detect_info[:, 0:4])
        confs = detect_info[:, 4]
        clss = detect_info[:, 5]
    print("do something with the reshaped results")

if __name__ == "__main__":
    image_path = 'data/images/zidane.jpg'
    image = cv2.imread(image_path)
    model = torch.hub.load('yolov5', 'custom', path='yolov5s.pt', source='local')
    model.conf = 0.6
    model.iou = 0.1
    main(image, model)

[Maioy97]

I put the yolov5 cloned project inside my project

      project
            |_ yolov5 
            |_ code.py

I really don't understand what might be causing this same environment same code sometimes works sometimes doesn't when it started working in my code I just re cloned my project from my repo and it worked (that repo had an old version of yolov5 in it) this time I'm trying to use it with a clean project with only the newly cloned yolov5 repo with a python file I just wrote

[glenn-jocher]

@Maioy97 please provide a fully reproducible code block, i.e. so I just need to run it and it works. You probably want to start with the git clone and cd commands etc as I showed above in https://github.com/ultralytics/yolov5/issues/9629#issuecomment-1322759810

[Maioy97]

here you got , I tried it on colab to be sure you can use it as is

!git clone https://github.com/ultralytics/yolov5  # clone
%cd yolov5
%pip install -qr requirements.txt  # install
%cd ../
!wget https://github.com/ultralytics/yolov5/releases/download/v6.2/yolov5s.pt

import sys
sys.path.insert(0, './yolov5')
from yolov5.utils.general import scale_boxes, xyxy2xywh
import torch
import cv2

def main(image, yolo_model):
    results = yolo_model(image, size=640)
    # logger.info("in_cam -- face detection end")
    for i, detect_info in enumerate(results.xyxy):
        print("detections have been detected")
        # print(det[:, :4])
        img1shape = image.shape[0:2]
        detected_coordinates = detect_info[:, :4]
        img0shape = image.shape
        detect_info[:, :4] = scale_boxes(img1shape, detected_coordinates, img0shape).round()
        xywhs = xyxy2xywh(detect_info[:, 0:4])
        confs = detect_info[:, 4]
        clss = detect_info[:, 5]
    print("do something with the reshaped results")

if __name__ == "__main__":
    image_path = 'yolov5/data/images/zidane.jpg'
    image = cv2.imread(image_path)
    model = torch.hub.load('yolov5', 'custom', path='yolov5s.pt', source='local')
    model.conf = 0.6
    model.iou = 0.1
    main(image, model)

[glenn-jocher]

@Maioy97 ok got it. I'm able to reproduce your error now. It seems like inference_mode() has been inherited by the results.xyxy tensor, which is kind of strange. In any case you can sidestep the problem by cloning the tensor, i.e. for i, detect_info in enumerate(results.xyxy.clone()):

This talks about the issue, says inference modes tensors remain immutable outside of inference mode, which is the problem you are seeing. https://pytorch.org/cppdocs/notes/inference_mode.html

[glenn-jocher]

@Maioy97 nevermind, results.xyxy is a list so that won't work. Try this:

    for i, xyxy_immutable in enumerate(results.xyxy):
        detect_info = xyxy_immutable.clone()

[Maioy97]

    for i, xyxy_immutable in enumerate(results.xyxy):
        detect_info = xyxy_immutable.clone()

Thank you, that seems to have worked

[glenn-jocher]

@Maioy97 glad to hear that it worked! If you have any other questions or issues, feel free to ask.