search

HZAI-ZJNU / Mamba-YOLO

the official pytorch implementation of “Mamba-YOLO：SSMs-based for Object Detection”
Apache License 2.0
135 stars 14 forks source link

Could you tell me how to use the yolo-manba-seg.yaml? #18

Open lijoe123 opened 3 days ago

lijoe123 commented 3 days ago

Hello, Could you tell me how to use the yolo-manba-seg.yaml? When i used the command :python mbyolo_train.py --task train --data ultralytics/cfg/datasets/coco123.yaml --config ultralytics/cfg/models/mamba-yolo/yolo-mamba-seg.yaml --amp --project work_dir/manba_yolo --name mabayolo

It occur this problem:

WARNING ⚠️ no model scale passed. Assuming scale='n'.
Traceback (most recent call last):
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 301, in __init__
    m.stride = torch.tensor([s / x.shape[-2] for x in forward(torch.zeros(1, ch, s, s))])  # forward
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in <lambda>
    forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward
    return self.predict(x, *args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict
    return self._predict_once(x, profile, visualize, embed)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once
    x = m(x)  # run
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward
    x = input + self.drop_path(self.op(self.norm(X1)))
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 186, in forward
    y = self.forward_core(x, channel_first=(self.d_conv > 1))
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 165, in forward_corev2
    x = cross_selective_scan(
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 192, in cross_selective_scan
    ys: torch.Tensor = selective_scan(
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 168, in selective_scan
    return SelectiveScan.apply(u, delta, A, B, C, D, delta_bias, delta_softplus, nrows, backnrows, ssoflex)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 98, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 125, in forward
    out, x, *rest = selective_scan_cuda_core.fwd(u, delta, A, B, C, D, delta_bias, delta_softplus, 1)
RuntimeError: Expected u.is_cuda() to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/music/Downloads/Mamba-YOLO-main/mbyolo_train.py", line 44, in <module>
    "train": YOLO(model_conf).train(**args),
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/models/yolo/model.py", line 23, in __init__
    super().__init__(model=model, task=task, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/engine/model.py", line 150, in __init__
    self._new(model, task=task, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/engine/model.py", line 219, in _new
    self.model = (model or self._smart_load("model"))(cfg_dict, verbose=verbose and RANK == -1)  # build model
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 380, in __init__
    super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 308, in __init__
    forward(torch.zeros(2, ch, s, s).to(torch.device('cuda')))])  # forward
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in <lambda>
    forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward
    return self.predict(x, *args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict
    return self._predict_once(x, profile, visualize, embed)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once
    x = m(x)  # run
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward
    x = input + self.drop_path(self.op(self.norm(X1)))
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/timm/layers/drop.py", line 179, in forward
    return drop_path(x, self.drop_prob, self.training, self.scale_by_keep)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/timm/layers/drop.py", line 162, in drop_path
    keep_prob = 1 - drop_prob
TypeError: unsupported operand type(s) for -: 'int' and 'str'
EthanW-coder commented 2 days ago

Hello, this issue comes from the fact that you are using the default parameters for multi-GPU training in mbyolo_train.py. If you are training with only one GPU, could you please change parser.add_argument('--device', default='0,1,2,3,4,5,6,7', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device'). argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')

lijoe123 commented 2 days ago

Hello, this issue comes from the fact that you are using the default parameters for multi-GPU training in mbyolo_train.py. If you are training with only one GPU, could you please change parser.add_argument('--device', default='0,1,2,3,4,5,6,7', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device'). argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')

Thank you for your answer. I am training the manba_yolo_detect. And i want to trained the manba_yolo_segment. So the command is as same as the detect task?

EthanW-coder commented 2 days ago

Hello, this issue comes from the fact that you are using the default parameters for multi-GPU training in mbyolo_train.py. If you are training with only one GPU, could you please change parser.add_argument('--device', default='0,1,2,3,4,5,6,7', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device'). argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')

Thank you for your answer. I am training the manba_yolo_detect. And i want to trained the manba_yolo_segment. So the command is as same as the detect task?

Yes, you just need to make sure your dataset is in yolo-seg format.

lijoe123 commented 2 days ago

Hello, this issue comes from the fact that you are using the default parameters for multi-GPU training in mbyolo_train.py. If you are training with only one GPU, could you please change parser.add_argument('--device', default='0,1,2,3,4,5,6,7', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') to parser.add_argument('--device', default='0', help='cuda device'). argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')

Thank you for your answer. I am training the manba_yolo_detect. And i want to trained the manba_yolo_segment. So the command is as same as the detect task?

Yes, you just need to make sure your dataset is in yolo-seg format.

Yes, of course. I had trained it in yolov9_seg

lijoe123 commented 2 days ago

I am sorry to tell you that, It still had the same problem, I had finish the ,manba_yolo_detect train. And i had finish yolov9_seg train. They are in the same segment dataset

WARNING ⚠️ no model scale passed. Assuming scale='n'.
Traceback (most recent call last):
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 301, in __init__
    m.stride = torch.tensor([s / x.shape[-2] for x in forward(torch.zeros(1, ch, s, s))])  # forward
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in <lambda>
    forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward
    return self.predict(x, *args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict
    return self._predict_once(x, profile, visualize, embed)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once
    x = m(x)  # run
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward
    x = input + self.drop_path(self.op(self.norm(X1)))
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 186, in forward
    y = self.forward_core(x, channel_first=(self.d_conv > 1))
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 165, in forward_corev2
    x = cross_selective_scan(
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 192, in cross_selective_scan
    ys: torch.Tensor = selective_scan(
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 168, in selective_scan
    return SelectiveScan.apply(u, delta, A, B, C, D, delta_bias, delta_softplus, nrows, backnrows, ssoflex)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 98, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 125, in forward
    out, x, *rest = selective_scan_cuda_core.fwd(u, delta, A, B, C, D, delta_bias, delta_softplus, 1)
RuntimeError: Expected u.is_cuda() to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/music/Downloads/Mamba-YOLO-main/mbyolo_train.py", line 44, in <module>
    "train": YOLO(model_conf).train(**args),
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/models/yolo/model.py", line 23, in __init__
    super().__init__(model=model, task=task, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/engine/model.py", line 150, in __init__
    self._new(model, task=task, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/engine/model.py", line 219, in _new
    self.model = (model or self._smart_load("model"))(cfg_dict, verbose=verbose and RANK == -1)  # build model
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 380, in __init__
    super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 308, in __init__
    forward(torch.zeros(2, ch, s, s).to(torch.device('cuda')))])  # forward
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in <lambda>
    forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward
    return self.predict(x, *args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict
    return self._predict_once(x, profile, visualize, embed)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once
    x = m(x)  # run
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/Downloads/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward
    x = input + self.drop_path(self.op(self.norm(X1)))
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/timm/layers/drop.py", line 179, in forward
    return drop_path(x, self.drop_prob, self.training, self.scale_by_keep)
  File "/home/music/miniconda3/envs/yolov10/lib/python3.9/site-packages/timm/layers/drop.py", line 162, in drop_path
    keep_prob = 1 - drop_prob
TypeError: unsupported operand type(s) for -: 'int' and 'str'
hafsa390 commented 1 day ago

Hi @lijoe123 did you train the mamba-yolo for detection with the coco dataset? Can it detect the objects during validation/test?

lijoe123 commented 1 day ago

Hi @lijoe123 did you train the mamba-yolo for detection with the coco dataset? Can it detect the objects during validation/test?

Yes, I can train it in detection. And i can detect the objects

hafsa390 commented 1 day ago

@lijoe123 Did you made any change in the loss function or anything in the provided code?

lijoe123 commented 1 day ago

No, I just change the dataset.

lijoe123 commented 1 day ago

So, Had you train it in the mamba_yolo_seg successfully?

hafsa390 commented 1 day ago

No. I was trying to perform detection. But can not detect the objects.