Docker GPU error - Githubissues

Im using the docker/Dockerfile image as my enviroment. Once the image is built I create the checkpoint folder and download the models needed for the demos. Finally I connect via:

docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmdetection3d/data mmdetection3d

When running demo/pcd_demo.py script I get the following error:

python demo/pcd_demo.py demo/data/kitti/kitti_000008.bin configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth
Use load_from_local loaderTraceback (most recent call last):
  File "demo/pcd_demo.py", line 41, in <module>
    main()
  File "demo/pcd_demo.py", line 28, in main
    result, data = inference_detector(model, args.pcd)
  File "/mmdetection3d/mmdet3d/apis/inference.py", line 115, in inference_detector
    result = model(return_loss=False, rescale=True, **data)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 95, in new_func
    return old_func(*args, **kwargs)
  File "/mmdetection3d/mmdet3d/models/detectors/base.py", line 60, in forward
    return self.forward_test(**kwargs)
  File "/mmdetection3d/mmdet3d/models/detectors/base.py", line 41, in forward_test
    return self.simple_test(points[0], img_metas[0], img[0], **kwargs)
  File "/mmdetection3d/mmdet3d/models/detectors/voxelnet.py", line 98, in simple_test
    x = self.extract_feat(points, img_metas)
  File "/mmdetection3d/mmdet3d/models/detectors/voxelnet.py", line 40, in extract_feat
    voxels, num_points, coors = self.voxelize(points)
  File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 182, in new_func
    return old_func(*args, **kwargs)
  File "/mmdetection3d/mmdet3d/models/detectors/voxelnet.py", line 55, in voxelize
    res_voxels, res_coors, res_num_points = self.voxel_layer(res)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 113, in forward
    self.max_num_points, max_voxels)
  File "/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 53, in forward
    coors_range, max_points, max_voxels, 3)
RuntimeError: CUDA error: no kernel image is available for execution on the device

When running demo/mono_det_demo.py I get another GPU problem:

python demo/mono_det_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.jpg demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525_mono3d.coco.json configs/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune.py checkpoints/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune_20210427_091419-35aaaad0.pth

/opt/conda/lib/python3.7/site-packages/mmcv/cnn/bricks/conv_module.py:107: UserWarning: ConvModule has norm and bias at the same time
  warnings.warn('ConvModule has norm and bias at the same time')
Use load_from_local loader
Error!
Error!
Error!
Error!
Error!
Error!
Error!
Error!

With the following code I test if the GPU is available:

import torch

torch.cuda.is_available()
True

open-mmlab / mmdetection3d

Docker GPU error #683