search

NVIDIA / MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
https://nvidia.github.io/MinkowskiEngine
Other
2.43k stars 360 forks source link

209 cudaErrorNoKernelImageForDevice no kernel image is available for execution on the device #587

Open Yangrlan opened 4 months ago

Yangrlan commented 4 months ago

Hi, I have installed the Minkowski Engine (ME) and successfully imported it in the python terminal. A simple case of creating sparse tensors using ME was also successfully run. But I met with the error: "cudaErrorNoKernelImageForDevice no kernel image is available for execution on the device" when I tested the codes using ME. The error is:

File "/home/haoxin/crossloc3d/core/val.py", line 28, in val
    metrics = eval(cfg, log, db_data_loader, q_data_loader, task)
  File "/home/haoxin/crossloc3d/core/eval.py", line 101, in eval
    embeddings = task.step(meta, data)
  File "/home/haoxin/crossloc3d/tasks/ours_me_task.py", line 95, in step
    for c in range(len(coords_lst))])
  File "/home/haoxin/crossloc3d/tasks/ours_me_task.py", line 95, in <listcomp>
    for c in range(len(coords_lst))])
  File "/home/haoxin/anaconda3/envs/crossloc/lib/python3.7/site-packages/MinkowskiEngine-0.5.4-py3.7-linux-x86_64.egg/MinkowskiEngine/MinkowskiSparseTensor.py", line 276, in __init__
    coordinates, features, coordinate_map_key
  File "/home/haoxin/anaconda3/envs/crossloc/lib/python3.7/site-packages/MinkowskiEngine-0.5.4-py3.7-linux-x86_64.egg/MinkowskiEngine/MinkowskiSparseTensor.py", line 304, in initialize_coordinates
    ) = self._manager.insert_and_map(coordinates, *coordinate_map_key.get_key())
  File "/home/haoxin/anaconda3/envs/crossloc/lib/python3.7/site-packages/MinkowskiEngine-0.5.4-py3.7-linux-x86_64.egg/MinkowskiEngine/MinkowskiCoordinateManager.py", line 179, in insert_and_map
    return self._manager.insert_and_map(coordinates, tensor_stride, string_id)
RuntimeError: CUDA error encountered at: /home/haoxin/crossloc3d/MinkowskiEngine/src/3rdparty/concurrent_unordered_map.cuh:595: 209 cudaErrorNoKernelImageForDevice no kernel image is available for execution on the device

Desktop (please complete the following information):

I have tried all the solutions concerning this problem, but none worked. Does anyone have any ideas about this problem? Thank you.

jkim50104 commented 4 months ago

Did you export TORCH_CUDA_ARCH_LIST while compiling?

Yangrlan commented 4 months ago

Thank you for your comments and it really works. I have ignored exporting the parameter TORCH_CUDA_ARCH_LIST when I compiled ME. And after entering the two commands in the terminal:

export TORCH_CUDA_ARCH_LIST="8.0"
export CXX=c++; export CUDA_HOME=/usr/local/cuda-11.1; python setup.py install --blas=openblas --force_cuda

the error disappeared when I tested the codes. Thank you again for your reply!