Closed sparro12 closed 3 years ago
I met with the same phenomenon. For me the problem is that: Numba currently does not support CUDA 11.2 (not until 0.53.0)
I have been using docker to downgrade my CUDA version....
Thank you very much. We have multiple version of CUDA on the server. We will try to downgrade to 10.2.
The following error shows up when running the evaluate.py script. I've included the command. It appears to be something with the iou file. The command earlier ran smoothly for the iou3d so it is not that. Looks like it would be not related to our setup. Thank you for any solutions in advance.
(KM3D) sparro12@instance-20210219-1359:~/KM3D/RTM3D$ python ./src/tools/kitti-object-eval-python/evaluate.py evaluate --label_path=/data/KITTI/training/label_2 --label_split_file /data/KITTI/ImageSets/val.txt --current_class=0,1,2 --coco=False --result_path=/data/KITTI/exp/results/data/ Traceback (most recent call last): File "./src/tools/kitti-object-eval-python/evaluate.py", line 4, in
from eval import get_official_eval_result, get_coco_eval_result
File "/home/sparro12/KM3D/RTM3D/src/tools/kitti-object-eval-python/eval.py", line 6, in
from rotate_iou import rotate_iou_gpu_eval
File "/home/sparro12/KM3D/RTM3D/src/tools/kitti-object-eval-python/rotate_iou.py", line 262, in
def rotate_iou_kernel_eval(N, K, dev_boxes, dev_query_boxes, dev_iou, criterion=-1):
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/decorators.py", line 136, in kernel_jit
targetoptions=targetoptions)
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/compiler.py", line 811, in init
self.compile(sigs[0])
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/compiler.py", line 952, in compile
kernel.bind()
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/compiler.py", line 576, in bind
self._func.get()
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/compiler.py", line 446, in get
ptx = self.ptx.get()
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/compiler.py", line 416, in get
self._extra_options)
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/cudadrv/nvvm.py", line 548, in llvm_to_ptx
ptx = cu.compile(opts)
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/cudadrv/nvvm.py", line 236, in compile
self._try_error(err, 'Failed to compile\n')
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/cudadrv/nvvm.py", line 254, in _try_error
self.driver.check_error(err, "%s\n%s" % (msg, self.get_log()))
File "/home/sparro12/miniconda3/envs/KM3D/lib/python3.6/site-packages/numba/cuda/cudadrv/nvvm.py", line 144, in check_error
raise exc
numba.cuda.cudadrv.error.NvvmError: Failed to compile