search

HZAI-ZJNU / Mamba-YOLO

the official pytorch implementation of “Mamba-YOLO：SSMs-based for Object Detection”
Apache License 2.0
227 stars 27 forks source link

RuntimeError: Expected u.is_cuda() to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.) During handling of the above exception, another exception occurred: RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. #26

Open xsa12345 opened 2 months ago

xsa12345 commented 2 months ago

(mambayolo) rjxy@rjxy-System-Product-Name:~/fyl/Mamba-YOLO-main$ python mbyolo_train.py --task train --data ultralytics/cfg/datasets/coco.yaml --config ultralytics/cfg/models/v8/Mamba-YOLO-L.yaml --amp --project ./output_dir/mscoco --name mambayolo_n WARNING ⚠️ no model scale passed. Assuming scale='L'. Traceback (most recent call last): File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 301, in init m.stride = torch.tensor([s / x.shape[-2] for x in forward(torch.zeros(1, ch, s, s))]) # forward File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward return self.predict(x, *args, kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict return self._predict_once(x, profile, visualize, embed) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once x = m(x) # run File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/container.py", line 215, in forward input = module(input) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward x = input + self.drop_path(self.op(self.norm(X1))) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 186, in forward y = self.forward_core(x, channel_first=(self.d_conv > 1)) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 165, in forward_corev2 x = cross_selective_scan( File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 192, in cross_selective_scan ys: torch.Tensor = selective_scan( File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 168, in selective_scan return SelectiveScan.apply(u, delta, A, B, C, D, delta_bias, delta_softplus, nrows, backnrows, ssoflex) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/autograd/function.py", line 539, in apply return super().apply(*args, *kwargs) # type: ignore[misc] File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/cuda/amp/autocast_mode.py", line 113, in decorate_fwd return fwd(args, kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 125, in forward out, x, *rest = selective_scan_cuda_core.fwd(u, delta, A, B, C, D, delta_bias, delta_softplus, 1) RuntimeError: Expected u.is_cuda() to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/rjxy/fyl/Mamba-YOLO-main/mbyolo_train.py", line 50, in "train": YOLO(model_conf).train(args), File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/models/yolo/model.py", line 23, in init super().init(model=model, task=task, verbose=verbose) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/engine/model.py", line 150, in init self._new(model, task=task, verbose=verbose) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/engine/model.py", line 219, in _new self.model = (model or self._smart_load("model"))(cfg_dict, verbose=verbose and RANK == -1) # build model File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 308, in init forward(torch.zeros(2, ch, s, s).to(torch.device('cuda')))]) # forward File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 299, in forward = lambda x: self.forward(x)[0] if isinstance(m, (Segment, Pose, OBB)) else self.forward(x) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 92, in forward return self.predict(x, *args, *kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 110, in predict return self._predict_once(x, profile, visualize, embed) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/tasks.py", line 131, in _predict_once x = m(x) # run File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/container.py", line 215, in forward input = module(input) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 382, in forward x = input + self.drop_path(self.op(self.norm(X1))) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 186, in forward y = self.forward_core(x, channel_first=(self.d_conv > 1)) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/mamba_yolo.py", line 165, in forward_corev2 x = cross_selective_scan( File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 192, in cross_selective_scan ys: torch.Tensor = selective_scan( File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 168, in selective_scan return SelectiveScan.apply(u, delta, A, B, C, D, delta_bias, delta_softplus, nrows, backnrows, ssoflex) File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/autograd/function.py", line 539, in apply return super().apply(args, kwargs) # type: ignore[misc] File "/home/rjxy/anaconda3/envs/mambayolo/lib/python3.10/site-packages/torch/cuda/amp/autocast_mode.py", line 113, in decorate_fwd return fwd(*args, *kwargs) File "/home/rjxy/fyl/Mamba-YOLO-main/ultralytics/nn/modules/common_utils_mbyolo.py", line 125, in forward out, x, rest = selective_scan_cuda_core.fwd(u, delta, A, B, C, D, delta_bias, delta_softplus, 1) RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

求大神指点