search

ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
51.26k stars 16.44k forks source link

Unable to predict using PyTorch Hub #9030

Closed MehulKhanna closed 2 years ago

MehulKhanna commented 2 years ago

Search before asking

YOLOv5 Component

PyTorch Hub

Bug

YOLOv5 🚀 2022-8-19 Python-3.10.4 torch-1.12.1+cpu CPU

Fusing layers... 
Model summary: 117 layers, 4203837 parameters, 0 gradients, 10.4 GFLOPs
Traceback (most recent call last):
  File "/home/mehul/Documents/Sign-Language-Interpreter/main.py", line 5, in <module>
    results = model("dataset/train/A/A1.jpg")
  File "/home/mehul/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/mehul/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 463, in forward
    b, ch, h, w = im.shape  # batch, channel, height, width
AttributeError: 'str' object has no attribute 'shape'

Environment

Minimal Reproducible Example

import torch

model = torch.hub.load("ultralytics/yolov5", "custom", "models/best.pt")

results = model("dataset/train/A/A1.jpg")
results.print()

here models/best.pt is a classification model

Additional

No response

Are you willing to submit a PR?

github-actions[bot] commented 2 years ago

👋 Hello @MehulKhanna, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

CI CPU testing

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.

glenn-jocher commented 2 years ago

@MehulKhanna thanks for the bug report!

Currently ClassificationModel objects are not wrapped by AutoShape, so they are simple models that require the traditional input transforms below. This would be a good idea to handle Classification and Detection models the same way when loaded through PyTorch Hub. I'll think about how to do this best.

https://github.com/ultralytics/yolov5/blob/1cd3e752def0ecbcb39a95d75e3c93fad3114ab9/utils/augmentations.py#L345-L347

UygarUsta99 commented 2 years ago

When using hub models as ultralytics cache,below error persists.

Screenshot from 2022-08-19 16-19-52

Minimal Reproducible Example

model = torch.hub.load('ultralytics/yolov5', 'custom', 'yv5m-cls-marka.pt')  # load from PyTorch Hub
result = model(some_image_array)

Bug

Exception: Can't get attribute 'ClassificationModel' on <module 'models.yolo' from '/home/ai/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py'>. Cache may be out of date, try `force_reload=True` or see https://docs.ultralytics.com/yolov5/tutorials/pytorch_hub_model_loading for help.

When using with local cache,I receive the error described in the issue.

glenn-jocher commented 2 years ago

@UygarUsta99 your cache is out of date. pass force_reload=True

glenn-jocher commented 2 years ago

@UygarUsta99 then you can pass a BCHW tensor of shape 1,3,224,224

UygarUsta99 commented 2 years ago

@UygarUsta99 your cache is out of date. pass force_reload=True

This solved my problem.I am able to feed bchw tensor and get predictions.Thanks.

Mehul343 commented 2 years ago

@UygarUsta99 then you can pass a BCHW tensor of shape 1,3,224,224

How can I convert the image to BCHW? I converted the image to CHW [3, 224, 224] and it still doesn't work.

UygarUsta99 commented 2 years ago

@UygarUsta99 then you can pass a BCHW tensor of shape 1,3,224,224

How can I convert the image to BCHW? I converted the image to CHW [3, 224, 224] and it still doesn't work.

Add [None] after your tensor such as img[None].

MehulKhanna commented 2 years ago 
import torch
from PIL import Image
from torchvision import transforms as T

IMAGENET_MEAN = 0.485, 0.456, 0.406
IMAGENET_STD = 0.229, 0.224, 0.225

model = torch.hub.load("ultralytics/yolov5", "custom", "models/best.pt")

def classify_transforms(size=224):
    return T.Compose([T.ToTensor(), T.Resize(size), T.CenterCrop(size), T.Normalize(IMAGENET_MEAN, IMAGENET_STD)])

image = "dataset/test/A_test.jpg"
image = Image.open(image)

transformations = classify_transforms()
convert_tensor = transformations(image)
convert_tensor = convert_tensor.unsqueeze(0)
print(convert_tensor.shape)

results = model(convert_tensor)
print(results)

This did it. I used tensor.unsqueeze(0) to add the batch.

gd1925 commented 2 years ago

Hi @MehulKhanna and @glenn-jocher, Is there any way to load the model without using Pytorch Hub? Can I do it this way - weights = '/path/to/custom/yolov5model.pt' model = DetectMultiBackend(weights, device='cpu', dnn=False, data='/path/to/related/yaml', fp16=False) stride, names, pt = model.stride, model.names, model.pt

I tried this way, but I get no detections..If I pass the same image to the detect.py, I get all possible detections.

glenn-jocher commented 2 years ago

@gd1925 👋 Hello! Thanks for asking about handling inference results. YOLOv5 🚀 PyTorch Hub models allow for simple model loading and inference in a pure python environment without using detect.py.

Simple Inference Example

This example loads a pretrained YOLOv5s model from PyTorch Hub as model and passes an image for inference. 'yolov5s' is the YOLOv5 'small' model. For details on all available models please see the README. Custom models can also be loaded, including custom trained PyTorch models and their exported variants, i.e. ONNX, TensorRT, TensorFlow, OpenVINO YOLOv5 models.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # yolov5n - yolov5x6 official model
#                                            'custom', 'path/to/best.pt')  # custom model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.
results.xyxy[0]  # im predictions (tensor)

results.pandas().xyxy[0]  # im predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

results.pandas().xyxy[0].value_counts('name')  # class counts (pandas)
# person    2
# tie       1

See YOLOv5 PyTorch Hub Tutorial for details.

Good luck 🍀 and let us know if you have any other questions!

ummagumm-a commented 1 year ago

Hello. I don't understand how you are able to pass a path into the model. It throws me an error when I do that. Please help. Below is the output that I get

import torch

model = torch.hub.load('ultralytics/yolov5', 'yolov5x', force_reload=True)

print(type(model))

model('https://ultralytics.com/images/zidane.jpg')
Downloading: "https://github.com/ultralytics/yolov5/zipball/master" to /home/ummagumma/.cache/torch/hub/master.zip
YOLOv5 🚀 2023-1-12 Python-3.10.8 torch-1.13.1+cu117 CUDA:0 (Tesla V100S-PCIE-32GB, 32510MiB)

Fusing layers... 
<class 'models.yolo.DetectionModel'>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[144], line 8
      4                        #path='yolo/yolov5/runs/train/night_new_data_u8_corrected/weights/best.pt', force_reload=True) 
      6 print(type(model))
----> 8 model('https://ultralytics.com/images/zidane.jpg')

File ~/anaconda3/envs/svarka_xray/lib/python3.10/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
   1190 # If we don't have any hooks, we want to skip the rest of the logic in
   1191 # this function, and just call forward.
   1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1193         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194     return forward_call(*input, **kwargs)
   1195 # Do not call functions when jit is used
   1196 full_backward_hooks, non_full_backward_hooks = [], []

File ~/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py:209, in DetectionModel.forward(self, x, augment, profile, visualize)
    207 if augment:
    208     return self._forward_augment(x)  # augmented inference, None
--> 209 return self._forward_once(x, profile, visualize)

File ~/.cache/torch/hub/ultralytics_yolov5_master/models/yolo.py:121, in BaseModel._forward_once(self, x, profile, visualize)
    119 if profile:
    120     self._profile_one_layer(m, x, dt)
--> 121 x = m(x)  # run
    122 y.append(x if m.i in self.save else None)  # save output
    123 if visualize:

File ~/anaconda3/envs/svarka_xray/lib/python3.10/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
   1190 # If we don't have any hooks, we want to skip the rest of the logic in
   1191 # this function, and just call forward.
   1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1193         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194     return forward_call(*input, **kwargs)
   1195 # Do not call functions when jit is used
   1196 full_backward_hooks, non_full_backward_hooks = [], []

File ~/.cache/torch/hub/ultralytics_yolov5_master/models/common.py:57, in Conv.forward(self, x)
     56 def forward(self, x):
---> 57     return self.act(self.bn(self.conv(x)))

File ~/anaconda3/envs/svarka_xray/lib/python3.10/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
   1190 # If we don't have any hooks, we want to skip the rest of the logic in
   1191 # this function, and just call forward.
   1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1193         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194     return forward_call(*input, **kwargs)
   1195 # Do not call functions when jit is used
   1196 full_backward_hooks, non_full_backward_hooks = [], []

File ~/anaconda3/envs/svarka_xray/lib/python3.10/site-packages/torch/nn/modules/conv.py:463, in Conv2d.forward(self, input)
    462 def forward(self, input: Tensor) -> Tensor:
--> 463     return self._conv_forward(input, self.weight, self.bias)

File ~/anaconda3/envs/svarka_xray/lib/python3.10/site-packages/torch/nn/modules/conv.py:459, in Conv2d._conv_forward(self, input, weight, bias)
    455 if self.padding_mode != 'zeros':
    456     return F.conv2d(F.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode),
    457                     weight, bias, self.stride,
    458                     _pair(0), self.dilation, self.groups)
--> 459 return F.conv2d(input, weight, bias, self.stride,
    460                 self.padding, self.dilation, self.groups)

TypeError: conv2d() received an invalid combination of arguments - got (str, Parameter, NoneType, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!str!, !Parameter!, !NoneType!, !tuple!, !tuple!, !tuple!, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!str!, !Parameter!, !NoneType!, !tuple!, !tuple!, !tuple!, int)
Sai-dinesh-k commented 1 year ago

that import error of classificationmodel not solved for me

ImportError Traceback (most recent call last) Input In [42], in <cell line: 1>() ----> 1 model = torch.hub.load('ultralytics/yolov5', 'custom', path='yolov5/runs/train/exp1/weights/best.pt', force_reload=True)

File ~\anaconda3\lib\site-packages\torch\hub.py:339, in load(repo_or_dir, model, *args, *kwargs) 336 if source == 'github': 337 repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose) --> 339 model = _load_local(repo_or_dir, model, args, **kwargs) 340 return model

File ~\anaconda3\lib\site-packages\torch\hub.py:368, in _load_local(hubconf_dir, model, *args, *kwargs) 365 hub_module = import_module(MODULE_HUBCONF, hubconf_path) 367 entry = _load_entry_from_hubconf(hub_module, model) --> 368 model = entry(args, **kwargs) 370 sys.path.remove(hubconf_dir) 372 return model

File ~/.cache\torch\hub\ultralytics_yolov5_master\hubconf.py:83, in custom(path, autoshape, _verbose, device) 81 def custom(path='path/to/model.pt', autoshape=True, _verbose=True, device=None): 82 # YOLOv5 custom or local model ---> 83 return _create(path, autoshape=autoshape, verbose=_verbose, device=device)

File ~/.cache\torch\hub\ultralytics_yolov5_master\hubconf.py:35, in _create(name, pretrained, channels, classes, autoshape, verbose, device) 33 from models.common import AutoShape, DetectMultiBackend 34 from models.experimental import attempt_load ---> 35 from models.yolo import ClassificationModel, DetectionModel, SegmentationModel 36 from utils.downloads import attempt_download 37 from utils.general import LOGGER, check_requirements, intersect_dicts, logging

ImportError: cannot import name 'ClassificationModel' from 'models.yolo' (C:\Users\kavil/.cache\torch\hub\ultralytics_yolov5_master\models\yolo.py)

Abhiramborige commented 1 year ago

@gd1925 👋 Hello! Thanks for asking about handling inference results. YOLOv5 🚀 PyTorch Hub models allow for simple model loading and inference in a pure python environment without using detect.py.

Simple Inference Example

This example loads a pretrained YOLOv5s model from PyTorch Hub as model and passes an image for inference. 'yolov5s' is the YOLOv5 'small' model. For details on all available models please see the README. Custom models can also be loaded, including custom trained PyTorch models and their exported variants, i.e. ONNX, TensorRT, TensorFlow, OpenVINO YOLOv5 models.

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')  # yolov5n - yolov5x6 official model
#                                            'custom', 'path/to/best.pt')  # custom model

# Images
im = 'https://ultralytics.com/images/zidane.jpg'  # or file, Path, URL, PIL, OpenCV, numpy, list

# Inference
results = model(im)

# Results
results.print()  # or .show(), .save(), .crop(), .pandas(), etc.
results.xyxy[0]  # im predictions (tensor)

results.pandas().xyxy[0]  # im predictions (pandas)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie

results.pandas().xyxy[0].value_counts('name')  # class counts (pandas)
# person    2
# tie       1

See YOLOv5 PyTorch Hub Tutorial for details.

Good luck 🍀 and let us know if you have any other questions!

# Inference
results = model(im)

In this code snippet, how to pass the parameters like size of image, confidence threshold, iou threshold, device to be used for prediction, maximum detections, etc....

The Autoshape class has the following signature:

class AutoShape(nn.Module):
    # YOLOv5 input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS
    conf = 0.25  # NMS confidence threshold
    iou = 0.45  # NMS IoU threshold
    agnostic = False  # NMS class-agnostic
    multi_label = False  # NMS multiple labels per box
    classes = None  # (optional list) filter by class, i.e. = [0, 15, 16] for COCO persons, cats and dogs
    max_det = 1000  # maximum number of detections per image
    amp = False  # Automatic Mixed Precision (AMP) inference

But, how to change the cong, iou, etc... while calling the results = model(im) snippet.

@smart_inference_mode()
    def forward(self, ims, size=640, augment=False, profile=False):

I am able to resize the image with this function in AutoShape, but not set iou, conf, max_det.

Please help me out if there is any other way to set those while prediction using yolov5 model and using basic pytorch code as mentioned in the quoted message

glenn-jocher commented 1 year ago

@Abhiramborige hello! To change the values for confidence threshold, IoU threshold, maximum detections, etc. while calling results = model(im) you can modify the default values set in the AutoShape class by creating an instance of AutoShape and passing in your desired values as arguments. For example:

from models.common import AutoShape

# create instance of AutoShape with desired values
autoshape = AutoShape(conf=0.3, iou=0.5, max_det=500)

# load model from hub and pass in autoshape as argument
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', autoshape=autoshape)

# pass image for inference
results = model(im)

By creating an instance of AutoShape with your desired values and passing it in as an argument when loading the model from hub, those values will be used during inference.

Hope this helps! Let us know if you have any further questions.

Abhiramborige commented 1 year ago

This is fine when we install the yolov5 repo locally and use it. I want to load the yolov5 with torch.hub.load('github repo') How can we do that. Thankyou for the response

Abhiramborige commented 1 year ago

And also, pycharm is throwing error.

Screenshot (66)

glenn-jocher commented 1 year ago

@Abhiramborige hello! Thank you for submitting an issue. To help us better understand the problem you're facing, could you provide more information about the error you're seeing in PyCharm? It would be helpful to see the full error message with any accompanying traceback. Additionally, have you tried running the same code outside of PyCharm, for example in a Jupyter notebook or a Python script from the command line? This could help narrow down whether the issue is specific to PyCharm or to your system/environment setup. Thank you!

Abhiramborige commented 1 year ago

Its telling that AutoShape doesnt expected to have the arguments. I havent tried in other platforms except Pycharm. I wanted to run with a Python Script.

glenn-jocher commented 1 year ago

Hello @Abhiramborige, thank you for providing more information about the issue you are facing. It looks like the error message in PyCharm is telling you that AutoShape does not expect to have the arguments you passed in.

When using torch.hub.load to load the YOLOv5 model from a GitHub repository, you can create an instance of AutoShape with your desired argument values and pass it in as a keyword argument during model loading. Here's an example:

from models.common import AutoShape
import torch

autoshape = AutoShape(conf=0.3, iou=0.5, max_det=500)

model = torch.hub.load(
    'ultralytics/yolov5', 
    'yolov5s', 
    autoshape=autoshape
)

results = model(im)

By creating an instance of AutoShape with your desired argument values and passing it in as a keyword argument during model loading with torch.hub.load, you should be able to set the values that you need for confidence threshold, IoU threshold, maximum detections, etc. during inference.

Hope this helps! Let us know if you have any further questions or concerns.

Abhiramborige commented 1 year ago

Great, thanks for that. And also I have one more doubt. I am actually seeing some difference in predictions returned by:

  1. predict.py in the yolov5 repo
  2. prediciton using torch.hub.load(model)(image) method. There are few predictions coming from 1st method which are not returned by the 2nd method.

PS: I am setting the hyperparameters while predicting as you mentioned using AutoShape constructor and passing the class to torch.hub.load. Still not able to rectify. Looking forward for your helpful response. :)

glenn-jocher commented 1 year ago

@Abhiramborige hello! Thank you for reaching out to us. It's great to hear you're using YOLOv5 for your projects!

Regarding the difference in predictions between using predict.py in the YOLOv5 repo and using torch.hub.load(model) method - it could be caused by differences in the input processing and post-processing steps. The two methods might be using slightly different code for loading the model and calling the necessary post-processing functions.

One thing to note is that the default settings for AutoShape class might be different between the two methods, so it's important to specify the same values for confidence, IoU threshold, maximum detections and other parameters in both methods to ensure a fair comparison.

If you're still experiencing differences in predictions despite setting these parameters exactly the same in both methods, please provide more details about the specific differences between the predicted results, along with any additional information about the images you're testing with and any sample code you're using to process the results. This could help us better understand the issue and provide you with more personalized assistance.

Thank you for your patience and continued interest in YOLOv5!

Abhiramborige commented 1 year ago

Thankyou for the answers. I would also like to know why the cuda usage graph in task manager is having peaks (90%)and valleys(0% usage) when i am training with GPU and HDD whereas the cuda usage graph is constant and varies little bit around 85 to 90% when training with GPU and SSD? Is SSD compulsory for a faster training process along with GPU ? Why my HDD usage is at 100% all the time when training with HDD and low usage (around 10-20%) when training with SSD. Thankyou.

glenn-jocher commented 1 year ago

@Abhiramborige hello!

The fluctuations in the CUDA usage graph that you see on your task manager could be due to a number of factors, including the size of your dataset, batch size, model complexity, number of iterations during training, and hardware and software configurations. When using an HDD, you may experience slower data access and loading times compared to using an SSD, which could cause your CUDA usage to drop and result in fluctuations over time. Using an SSD can help mitigate these bottlenecks, as data access and loading times are typically faster. However, using an SSD is not necessarily compulsory for faster training – it ultimately depends on your specific hardware and software setup and the size and complexity of your dataset and model.

Regarding the 100% HDD usage when training with an HDD – this could be due to the slower read/write speeds of the HDD, which can bottleneck the training process and cause your system to prioritize HDD usage. When training with an SSD, the faster read/write speeds reduce the bottleneck in data access and loading, and as a result, your system may prioritize other resources, such as your GPU(s), resulting in lower HDD usage.

Overall, there are many factors that can affect the performance of your training process, including hardware, software, and data considerations. It is important to optimize your setup as best as possible and choose the appropriate resources to ensure optimal performance. We hope this helps - if you have any other questions or concerns, please let us know.