Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

[bryanbocao]

When I tried

Torch-Pruning/benchmarks/prunability$ python3 yolov8_pruning.py

I got

YOLOv8m summary (fused): 235 layers, 21965887 parameters, 58816 gradients, 67.2 GFLOPs
val: Scanning /home/brcao/Repos/Torch-Pruning/benchmarks/prunability/datasets/coco128/labels/train2017.cache...
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 128/1
                   all        128        929     0.0312     0.0283     0.0312     0.0193
                person        128        254      0.246      0.063      0.155     0.0807
               bicycle        128          6          0          0          0          0
                   car        128         46          0          0          0          0
            motorcycle        128          5          0          0          0          0
              airplane        128          6          0          0          0          0
                   bus        128          7          0          0          0          0
                 train        128          3          0          0          0          0
                 truck        128         12          0          0          0          0
                  boat        128          6          0          0          0          0
         traffic light        128         14          0          0          0          0
             stop sign        128          2          0          0          0          0
                 bench        128          9          0          0          0          0
                  bird        128         16          0          0          0          0
                   cat        128          4          0          0          0          0
                   dog        128          9          0          0          0          0
                 horse        128          2          0          0          0          0
              elephant        128         17          0          0          0          0
                  bear        128          1          0          0          0          0
                 zebra        128          4          0          0          0          0
               giraffe        128          9      0.125      0.111       0.17     0.0531
              backpack        128          6          0          0          0          0
              umbrella        128         18      0.333     0.0556      0.213      0.085
               handbag        128         19          0          0          0          0
                   tie        128          7      0.143      0.143      0.206     0.0824
              suitcase        128          4          0          0          0          0
               frisbee        128          5          0          0          0          0
                  skis        128          1          0          0          0          0
             snowboard        128          7          0          0          0          0
           sports ball        128          6          0          0          0          0
                  kite        128         10          0          0          0          0
          baseball bat        128          4          0          0          0          0
        baseball glove        128          7          0          0          0          0
            skateboard        128          5          0          0          0          0
         tennis racket        128          7          0          0          0          0
                bottle        128         18          0          0          0          0
            wine glass        128         16          0          0          0          0
                   cup        128         36      0.154     0.0556     0.0886     0.0627
                  fork        128          6          0          0          0          0
                 knife        128         16          0          0          0          0
                 spoon        128         22          0          0          0          0
                  bowl        128         28      0.278      0.179      0.285      0.234
                banana        128          1          0          0          0          0
              sandwich        128          2          0          0          0          0
                orange        128          4     0.0116        0.5     0.0169    0.00643
              broccoli        128         11          0          0          0          0
                carrot        128         24          0          0          0          0
               hot dog        128          2          0          0          0          0
                 pizza        128          5          0          0          0          0
                 donut        128         14        0.4      0.429      0.541      0.449
                  cake        128          4          0          0          0          0
                 chair        128         35          0          0          0          0
                 couch        128          6          0          0          0          0
          potted plant        128         14      0.222      0.143      0.191     0.0611
                   bed        128          3          0          0          0          0
          dining table        128         13          0          0          0          0
                toilet        128          2          0          0          0          0
                    tv        128          2          0          0          0          0
                laptop        128          3          0          0          0          0
                 mouse        128          2          0          0          0          0
                remote        128          8          0          0          0          0
            cell phone        128          8          0          0          0          0
             microwave        128          3          0          0          0          0
                  oven        128          5          0          0          0          0
                  sink        128          6          0          0          0          0
          refrigerator        128          5          0          0          0          0
                  book        128         29          0          0          0          0
                 clock        128          9        0.3      0.333      0.351      0.256
                  vase        128          2          0          0          0          0
              scissors        128          1          0          0          0          0
            teddy bear        128         21          0          0          0          0
            toothbrush        128          5          0          0          0          0
Speed: 0.5ms preprocess, 7.4ms inference, 0.0ms loss, 0.4ms postprocess per image
Results saved to runs/detect/step_1_pre_val
After pruning iter 2: MACs=33.692628 G, #Params=21.981122 M, mAP=0.019298016681157742, speed up=1.174367662860849
Ultralytics YOLOv8.0.117 🚀 Python-3.8.10 torch-2.0.0 CUDA:0 (NVIDIA A100-PCIE-40GB, 40514MiB)
yolo/engine/trainer: task=detect, mode=train, model=None, data=coco128.yaml, epochs=10, patience=50, batch=16, imgsz=640, save=True, save_period=-1, cache=False, device=None, workers=8, project=None, name=step_1_finetune, exist_ok=False, pretrained=False, optimizer=auto, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=0, resume=False, amp=True, fraction=1.0, profile=False, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, show=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, vid_stride=1, line_width=None, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, boxes=True, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0, cfg=None, v5loader=False, tracker=botsort.yaml, save_dir=runs/detect/step_1_finetune
TensorBoard: Start with 'tensorboard --logdir runs/detect/step_1_finetune', view at http://localhost:6006/
AMP: running Automatic Mixed Precision (AMP) checks with YOLOv8n...
AMP: checks passed ✅
train: Scanning /home/brcao/Repos/Torch-Pruning/benchmarks/prunability/datasets/coco128/labels/train2017.cache..
val: Scanning /home/brcao/Repos/Torch-Pruning/benchmarks/prunability/datasets/coco128/labels/train2017.cache...
Plotting labels to runs/detect/step_1_finetune/labels.jpg...
^@optimizer: AdamW(lr=0.000119, momentum=0.9) with parameter groups 85 weight(decay=0.0), 92 weight(decay=0.0005), 91 bias(decay=0.0)
Image sizes 640 train, 640 val
Using 6 dataloader workers
Logging results to runs/detect/step_1_finetune
Starting training for 10 epochs...

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
  0%|          | 0/8 [00:04<?, ?it/s]
Traceback (most recent call last):
  File "yolov8_pruning.py", line 392, in <module>
    prune(args)
  File "yolov8_pruning.py", line 353, in prune
    model.train_v2(pruning=True, **pruning_cfg)
  File "yolov8_pruning.py", line 267, in train_v2
    self.trainer.train()
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/yolo/engine/trainer.py", line 192, in train
    self._do_train(world_size)
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/yolo/engine/trainer.py", line 330, in _do_train
    self.loss, self.loss_items = self.model(batch)
  File "/usr/lib/python3/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/nn/tasks.py", line 44, in forward
    return self.loss(x, *args, **kwargs)
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/nn/tasks.py", line 213, in loss
    return self.criterion(self.predict(batch['img']) if preds is None else preds, batch)
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/yolo/utils/loss.py", line 153, in __call__
    pred_bboxes = self.bbox_decode(anchor_points, pred_distri)  # xyxy, (b, h*w, 4)
  File "/home/brcao/.local/lib/python3.8/site-packages/ultralytics/yolo/utils/loss.py", line 126, in bbox_decode
    pred_dist = pred_dist.view(b, a, 4, c // 4).softmax(3).matmul(self.proj.type(pred_dist.dtype))
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat in method wrapper_CUDA_addmv_)

[VainF]

Hi @bryanbocao, please try the old version of Yolov8.

git clone https://github.com/ultralytics/ultralytics.git 
cp yolov8_pruning.py ultralytics/
cd ultralytics 
git checkout 44c7c3514d87a5e05cfb14dba5a3eeb6eb860e70 # for compatibility

[bryanbocao]

@VainF Solved. Thanks!

[wujianfei5201314]

@VainF Solved. Thanks!

Have you tried it on the new version?

[wujianfei5201314]

Hi @bryanbocao, please try the old version of Yolov8.

git clone https://github.com/ultralytics/ultralytics.git 
cp yolov8_pruning.py ultralytics/
cd ultralytics 
git checkout 44c7c3514d87a5e05cfb14dba5a3eeb6eb860e70 # for compatibility

Have you tried it on the new version?

[bryanbocao]

@wujianfei5201314 Thanks for the info. I may try that if I use it later.

[Zzzxi]

嗨，请尝试旧版本的 Yolov8。

git clone https://github.com/ultralytics/ultralytics.git 
cp yolov8_pruning.py ultralytics/
cd ultralytics 
git checkout 44c7c3514d87a5e05cfb14dba5a3eeb6eb860e70 # for compatibility

你在新版本上试过吗？

May I ask if you have implemented it on the new version?

[Zzzxi]

解决。谢谢！

Can you only fix this error by downgrading the version?