search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.5k stars 9.44k forks source link

CUDA error: no kernel image is available for execution on the device #2389

Closed jonas154 closed 4 years ago

jonas154 commented 4 years ago

Hi there,

I tried installing your framework on another laptop, and facing now the following problems:

Traceback (most recent call last): File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/few_shot_learning/siamese_net.py", line 452, in <module> graph = hl.build_graph(model, x) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/hiddenlayer/hiddenlayer/graph.py", line 143, in build_graph import_graph(g, model, args) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/hiddenlayer/hiddenlayer/pytorch_builder.py", line 70, in import_graph trace, out = torch.jit._get_trace_graph(model, args) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/jit/__init__.py", line 277, in _get_trace_graph outs = ONNXTracedModule(f, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__ result = self.forward(*input, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/jit/__init__.py", line 360, in forward self._force_outplace, File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/jit/__init__.py", line 347, in wrapper outs.append(self.inner(*trace_inputs)) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/nn/modules/module.py", line 530, in __call__ result = self._slow_forward(*input, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/nn/modules/module.py", line 516, in _slow_forward result = self.forward(*input, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/few_shot_learning/siamese_net.py", line 406, in forward out = self.conv1(x) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/nn/modules/module.py", line 530, in __call__ result = self._slow_forward(*input, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/envs/lib/python3.7/site-packages/torch/nn/modules/module.py", line 516, in _slow_forward result = self.forward(*input, **kwargs) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/mmdetection/mmdet/ops/dcn/deform_conv.py", line 281, in forward self.dilation, self.groups, self.deformable_groups) File "/home/student/GIT_Repos/mm_wave_radar/machine_learning_approach/mmdetection/mmdet/ops/dcn/deform_conv.py", line 56, in forward cur_im2col_step) RuntimeError: CUDA error: no kernel image is available for execution on the device (launch_kernel at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/native/cuda/Loops.cuh:103) [...] Thats my envirionment: TorchVision: 0.5.0 OpenCV: 4.2.0 MMCV: 0.4.2 MMDetection: 1.1.0+336b101 MMDetection Compiler: GCC 7.5 MMDetection CUDA Compiler: 10.2

Have you an idea, how to fix this issue?

ZwwWayne commented 4 years ago

What is the detail information of the Pytorch you installed? Could you follow the Error Template and use the collect_env.py to provide more information about the environment?

zimenglan-sysu-512 commented 4 years ago

i meet this RuntimeError: CUDA error: no kernel image is available for execution on the device (ROIAlignForwardV2Laucher at mmdet/ops/roi_align/src/cuda/roi_align_kernel_v2.cu:305) Have you an idea, how to fix this issue?

ZwwWayne commented 4 years ago

i meet this RuntimeError: CUDA error: no kernel image is available for execution on the device (ROIAlignForwardV2Laucher at mmdet/ops/roi_align/src/cuda/roi_align_kernel_v2.cu:305) Have you an idea, how to fix this issue?

Please create a new issue following the error template if you still meet the bug and do not know how to solve it.