sanjaiiv04 commented 1 year ago

Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I am trying to use yolov5 for semantic segmentation and I wanted to use torch.hub.load to load the model to run inferences. I was able to download object detection models but I was not able to use torch to download segmentation models. So I tried to train a yolov5 segmentation model by using the coco dataset and the pre trained weight: 'yolov5s-seg.pt'. But I got the following error:

COMMAND: python3 train.py --data coco128.yaml --weights yolov5s-seg.pt --img 640 --batch 16 --epochs 10

OUTPUT: train: weights=yolov5s-seg.pt, cfg=, data=coco128.yaml, hyp=data/hyps/hyp.scratch-low.yaml, epochs=10, batch_size=16, imgsz=640, rect=False, resume=False, nosave=False, noval=False, noautoanchor=False, noplots=False, evolve=None, bucket=, cache=None, image_weights=False, device=, multi_scale=False, single_cls=False, optimizer=SGD, sync_bn=False, workers=8, project=runs/train, name=exp, exist_ok=False, quad=False, cos_lr=False, label_smoothing=0.0, patience=100, freeze=[0], save_period=-1, seed=0, local_rank=-1, entity=None, upload_dataset=False, bbox_interval=-1, artifact_alias=latest github: up to date with https://github.com/ultralytics/yolov5 ✅ YOLOv5 🚀 v7.0-240-g84ec8b5 Python-3.8.10 torch-2.1.0+cpu CPU

hyperparameters: lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=0.05, cls=0.5, cls_pw=1.0, obj=1.0, obj_pw=1.0, iou_t=0.2, anchor_t=4.0, fl_gamma=0.0, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0 Comet: run 'pip install comet_ml' to automatically track and visualize YOLOv5 🚀 runs in Comet TensorBoard: Start with 'tensorboard --logdir runs/train', view at http://localhost:6006/ Downloading https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s-seg.pt to yolov5s-seg.pt... 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14.9M/14.9M [00:06<00:00, 2.33MB/s]

             from  n    params  module                                  arguments

0 -1 1 1 -1 1 2 -1 1 3 -1 1 4 5 6 7 8 9 10 11 -1 1 12 [-1, 6] 1 13 14 15 -1 1 16 [-1, 4] 1 17 18 19 [-1, 14] 1 20 21 22 [-1, 10] 1 23 24 [17, 20, 23] 1 Model summary: 3520 models.common.Conv [3, 32, 6, 2, 2]
18560 models.common.Conv [32, 64, 3, 2]
18816 models.common.C3 [64, 64, 1]
73984 models.common.Conv [64, 128, 3, 2]
-1 2 115712 models.common.C3 [128, 128, 2]
-1 1 295424 models.common.Conv [128, 256, 3, 2]
-1 3 625152 models.common.C3 [256, 256, 3]
-1 1 1180672 models.common.Conv [256, 512, 3, 2]
-1 1 1182720 models.common.C3 [512, 512, 1]
-1 1 656896 models.common.SPPF [512, 512, 5]
-1 1 131584 models.common.Conv [512, 256, 1, 1]
0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
0 models.common.Concat [1]
-1 1 361984 models.common.C3 [512, 256, 1, False]
-1 1 33024 models.common.Conv [256, 128, 1, 1]
0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
0 models.common.Concat [1]
-1 1 90880 models.common.C3 [256, 128, 1, False]
-1 1 147712 models.common.Conv [128, 128, 3, 2]
0 models.common.Concat [1]
-1 1 296448 models.common.C3 [256, 256, 1, False]
-1 1 590336 models.common.Conv [256, 256, 3, 2]
0 models.common.Concat [1]
-1 1 1182720 models.common.C3 [512, 512, 1, False]
615133 models.yolo.Segment [80, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], 32, 128, [128, 256, 512]] 225 layers, 7621277 parameters, 7621277 gradients, 26.6 GFLOPs

Transferred 367/367 items from yolov5s-seg.pt optimizer: SGD(lr=0.01) with parameter groups 60 weight(decay=0.0), 63 weight(decay=0.0005), 63 bias train: Scanning /home/kukahtic/Sanjaii/datasets/coco128/labels/train2017.cache... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [00:00<?, ?it/s] val: Scanning /home/kukahtic/Sanjaii/datasets/coco128/labels/train2017.cache... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [00:00<?, ?it/s]

AutoAnchor: 4.27 anchors/target, 0.994 Best Possible Recall (BPR). Current anchors are a good fit to dataset ✅ Plotting labels to runs/train/exp/labels.jpg... Image sizes 640 train, 640 val Using 8 dataloader workers Logging results to runs/train/exp Starting training for 10 epochs...

  Epoch    GPU_mem   box_loss   obj_loss   cls_loss  Instances       Size

0%| | 0/8 [00:04<?, ?it/s] Traceback (most recent call last): File "train.py", line 647, in main(opt) File "train.py", line 536, in main train(opt.hyp, opt, device, callbacks) File "train.py", line 318, in train loss, loss_items = compute_loss(pred, targets.to(device)) # loss scaled by batch_size File "/home/kukahtic/Sanjaii/yolov5/utils/loss.py", line 125, in call tcls, tbox, indices, anchors = self.build_targets(p, targets) # targets File "/home/kukahtic/Sanjaii/yolov5/utils/loss.py", line 198, in build_targets anchors, shape = self.anchors[i], p[i].shape AttributeError: 'list' object has no attribute 'shape'.

How to resolve this? I am scared to change anything in the original script as it might completely affect the training process.

Additional

No response

github-actions[bot] commented 1 year ago

👋 Hello @sanjaiiv04, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

glenn-jocher commented 1 year ago

@sanjaiiv04 hi there,

Thank you for reaching out with your question. It seems that you are encountering an error while trying to train a YOLOv5 segmentation model using the torch.hub.load function. The error message suggests that there is an issue with the shape of the anchors.

To resolve this issue, you may need to check the anchor values and ensure that they have the correct shape. You can refer to the YOLOv5 documentation or the train.py script for more information on how the anchors should be formatted.

However, please note that I understand your concern about making changes to the original script. If you are unsure about modifying the script, I would recommend seeking assistance from the YOLOv5 community or the developers themselves. They will be in a better position to guide you on how to resolve this issue while minimizing the impact on the training process.

I hope this helps. Good luck with your YOLOv5 segmentation model!

sanjaiiv04 commented 1 year ago

@glenn-jocher Hey, the thing is I tried to use the command line to run the segmentation model and it works perfectly fine. The issue happens only when I am trying to use the trained model inside the code using torch.hub.load. When I try the following error comes: model=torch.hub.load('ultralytics/yolov5','yolov5s-seg.pt') it tells me there is exist no file like that in the hub. But when I use it via the yolov5 repository it works fine. How do you think I can use it in the torch.load function? If not how do you think I can use the segmentation model inside a script rather than the command line interface? The purpose of using this segmentation is to segment out people in real time via an externally connected camera. I need to be able to run the script and segment out people in the frame and to do that I need to load the model. Here is the code I did for object detection using the yolov5s.pt model. I am using an Intel Realsense D435 camera:

import cv2 import numpy as np import torch import pyrealsense2 as rs

model = torch.hub.load('ultralytics/yolov5', 'custom',path="/home/kukahtic/Sanjaii/yolov5s-seg.pt" ,force_reload=True, trust_repo=True) # or yolov5n - yolov5x6 or custom

model = torch.hub.load('ultralytics/yolov5', 'yolov5s',force_reload=True, trust_repo=True)

Configure depth and color streams

pipeline = rs.pipeline() config = rs.config()

Get device product line for setting a supporting resolution

pipeline_wrapper = rs.pipeline_wrapper(pipeline) pipeline_profile = config.resolve(pipeline_wrapper) device = pipeline_profile.get_device() device_product_line = str(device.get_info(rs.camera_info.product_line))

found_rgb = False for s in device.sensors: if s.get_info(rs.camera_info.name) == 'RGB Camera': found_rgb = True break if not found_rgb: print("The demo requires Depth camera with Color sensor") exit(0)

config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)

if device_product_line == 'L500': config.enable_stream(rs.stream.color, 960, 540, rs.format.bgr8, 30) else: config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

Start streaming

pipeline.start(config)

try: while True:

    # Wait for a coherent pair of frames: depth and color
    frames = pipeline.wait_for_frames()
    depth_frame = frames.get_depth_frame()
    color_frame = frames.get_color_frame()
    if not depth_frame or not color_frame:
        continue

    # Convert images to numpy arrays
    depth_image = np.asanyarray(depth_frame.get_data())
    color_image = np.asanyarray(color_frame.get_data())
    results = model(color_image)

    # Draw bounding boxes on the color frame
    result_image = results.render()[0]
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.05), cv2.COLORMAP_JET)

    depth_colormap_dim = depth_colormap.shape
    color_colormap_dim = color_image.shape

    # If depth and color resolutions are different, resize color image to match depth image for display
    if depth_colormap_dim != color_colormap_dim:
        resized_color_image = cv2.resize(color_image, dsize=(depth_colormap_dim[1], depth_colormap_dim[0]), interpolation=cv2.INTER_AREA)
        images = np.hstack((resized_color_image, depth_colormap))
    else:
        images = np.hstack((color_image, depth_colormap))

    # Show images
    cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)
    cv2.imshow('RealSense', images)
    cv2.waitKey(1)
    if cv2.waitKey(10) & 0xFF=='q':
        break

finally:

# Stop streaming
pipeline.stop(). I need to do the same with the segmentation model. How to do that?

glenn-jocher commented 1 year ago

@sanjaiiv04 hey, have you checked that the 'yolov5s-seg.pt' model exists in the ultralytics/yolov5 repository's hub directory? When working with custom models, it's best to verify that the model's file path is accurate. Also, ensure you're using the correct method to retrieve segmentation results. If you're looking to use the segmentation model within a script for real-time inference with an external camera, consider referring to the official YOLOv5 documentation for detailed guidance on integrating the segmentation model into your application.

github-actions[bot] commented 11 months ago

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

Docs: https://docs.ultralytics.com
HUB: https://hub.ultralytics.com
Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

ultralytics / yolov5

YOLOV5 segmentation model not working in torch.hub.load #12338