windows10 mmdet inference gives error on mmcv-full v1.4.2 (py3.9, cu113, mmdet@master)

[fcakyon]

steps to reproduce:

• setup env:

conda create -n mmlab python=3.9 -y 
conda activate mmlab 

conda install pytorch=1.10.0 torchvision=0.11.1 cudatoolkit=11.3 -c pytorch -y
pip install mmcv-full==1.4.2 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html

pip install git+https://github.com/open-mmlab/mmdetection.git@master

• confirm cuda and pytorch installation:

# Check Pytorch installation
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())

# Check MMDetection installation
import mmdet
print(mmdet.__version__)

# Check mmcv installation
from mmcv.ops import get_compiling_cuda_version, get_compiler_version
print(get_compiling_cuda_version())
print(get_compiler_version())

1.10.0 True
2.19.1
11.3
MSVC 192930137

• confirm mmcv installation:

python -c "from mmcv.ops import RoIPool"

no error

• clone mmdet repo for config files:

git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection

• download mmdet weight:

import urllib.request
from pathlib import Path

def download_from_url(from_url: str, to_path: str):
    Path(to_path).parent.mkdir(parents=True, exist_ok=True)
    urllib.request.urlretrieve(
        from_url,
        to_path,
    )

from_url='https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_tiny_8x8_300e_coco/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth'
to_path='weight.pth'
download_from_url(from_url, to_path)

• attempt to perform inference:

from mmdet.apis import inference_detector, init_detector

# Choose to use a config and initialize the detector
config = 'configs/yolox/yolox_tiny_8x8_300e_coco.py'
# Setup a checkpoint file to load
checkpoint = 'weight.pth'
# initialize the detector
model = init_detector(config, checkpoint, device='cuda:0')
# Use the detector to do inference
img = 'demo/demo.jpg'
result = inference_detector(model, img)

error:

load checkpoint from local path: weight.pth
C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\torch\functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ..\aten\src\ATen\native\TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py:284: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ..\torch\csrc\utils\tensor_new.cpp:201.)
  flatten_bboxes[..., :4] /= flatten_bboxes.new_tensor(
Traceback (most recent call last):
  File "C:\Users\xxx\dev\mmdetection\tools\perform_inference.py", line 11, in <module>
    result = inference_detector(model, img)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\apis\inference.py", line 147, in inference_detector
    results = model(return_loss=False, rescale=True, **data)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\runner\fp16_utils.py", line 98, in new_func
    return old_func(*args, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\base.py", line 174, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "C:\Users\FxxxA\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\base.py", line 147, in forward_test
    return self.simple_test(imgs[0], img_metas[0], **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\single_stage.py", line 102, in simple_test
    results_list = self.bbox_head.simple_test(
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\base_dense_head.py", line 360, in simple_test
    return self.simple_test_bboxes(feats, img_metas, rescale=rescale)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\dense_test_mixins.py", line 37, in simple_test_bboxes
    results_list = self.get_bboxes(
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py", line 294, in get_bboxes
    self._bboxes_nms(cls_scores, bboxes, score_factor, cfg))
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py", line 321, in _bboxes_nms
    dets, keep = batched_nms(bboxes, scores, labels, cfg.nms)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 307, in batched_nms
    dets, keep = nms_op(boxes_for_nms, scores, **nms_cfg_)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\utils\misc.py", line 340, in new_func
    output = old_func(*args, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 171, in nms
    inds = NMSop.apply(boxes, scores, iou_threshold, offset,
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 26, in forward
    inds = ext_module.nms(
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.

[zhouzaida]

Hi, what is your graphics card?

[fcakyon]

@zhouzaida its rtx3060 (compute capability 8.6)

[zhouzaida]

Could you try to install mmcv-full==1.4.1?

[fcakyon]

Could you try to install mmcv-full==1.4.1?

@zhouzaida

• setup env with mmcv-full==1.4.1:

conda create -n mmlab python=3.9 -y 
conda activate mmlab 

conda install pytorch=1.10.0 torchvision=0.11.1 cudatoolkit=11.3 -c pytorch -y
pip install mmcv-full==1.4.1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html

pip install git+https://github.com/open-mmlab/mmdetection.git@master

• confirm cuda and pytorch installation:

# Check Pytorch installation
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())

# Check MMDetection installation
import mmdet
print(mmdet.__version__)

# Check mmcv installation
from mmcv.ops import get_compiling_cuda_version, get_compiler_version
print(get_compiling_cuda_version())
print(get_compiler_version())

1.10.0 True
2.19.1
11.3
MSVC 192930137

• confirm mmcv installation:

python -c "from mmcv.ops import RoIPool"

no error

• clone mmdet repo for config files:

git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection

• download mmdet weight:

import urllib.request
from pathlib import Path

def download_from_url(from_url: str, to_path: str):
    Path(to_path).parent.mkdir(parents=True, exist_ok=True)
    urllib.request.urlretrieve(
        from_url,
        to_path,
    )

from_url='https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_tiny_8x8_300e_coco/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth'
to_path='weight.pth'
download_from_url(from_url, to_path)

• attempt to perform inference:

from mmdet.apis import inference_detector, init_detector

# Choose to use a config and initialize the detector
config = 'configs/yolox/yolox_tiny_8x8_300e_coco.py'
# Setup a checkpoint file to load
checkpoint = 'weight.pth'
# initialize the detector
model = init_detector(config, checkpoint, device='cuda:0')
# Use the detector to do inference
img = 'demo/demo.jpg'
result = inference_detector(model, img)

same error:

load checkpoint from local path: weight.pth
C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\torch\functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ..\aten\src\ATen\native\TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py:284: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ..\torch\csrc\utils\tensor_new.cpp:201.)
  flatten_bboxes[..., :4] /= flatten_bboxes.new_tensor(
Traceback (most recent call last):
  File "C:\Users\xxx\dev\mmdetection\tools\perform_inference.py", line 11, in <module>
    result = inference_detector(model, img)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\apis\inference.py", line 147, in inference_detector
    results = model(return_loss=False, rescale=True, **data)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\runner\fp16_utils.py", line 98, in new_func
    return old_func(*args, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\base.py", line 174, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "C:\Users\FxxxA\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\base.py", line 147, in forward_test
    return self.simple_test(imgs[0], img_metas[0], **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\detectors\single_stage.py", line 102, in simple_test
    results_list = self.bbox_head.simple_test(
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\base_dense_head.py", line 360, in simple_test
    return self.simple_test_bboxes(feats, img_metas, rescale=rescale)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\dense_test_mixins.py", line 37, in simple_test_bboxes
    results_list = self.get_bboxes(
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py", line 294, in get_bboxes
    self._bboxes_nms(cls_scores, bboxes, score_factor, cfg))
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmdet\models\dense_heads\yolox_head.py", line 321, in _bboxes_nms
    dets, keep = batched_nms(bboxes, scores, labels, cfg.nms)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 307, in batched_nms
    dets, keep = nms_op(boxes_for_nms, scores, **nms_cfg_)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\utils\misc.py", line 340, in new_func
    output = old_func(*args, **kwargs)
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 171, in nms
    inds = NMSop.apply(boxes, scores, iou_threshold, offset,
  File "C:\Users\xxx\Miniconda3\envs\xxx\lib\site-packages\mmcv\ops\nms.py", line 26, in forward
    inds = ext_module.nms(
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.

[fcakyon]

@zhouzaida maybe your precompiled mmcv windows binaries are not compiled for compute capability 8_6, might it be the reason? Have you tried running the pre-compiled mmcv binaries on RTX3XXX card on windows?

Refer to for the explanation: https://github.com/limbo018/DREAMPlace/issues/56#issue-1087715672

[fcakyon]

Based on this comment I am now updating my Nvidia Driver from 461.72 to 472.84: https://github.com/google/jax/issues/5723#issuecomment-838456737 but still having the same error.

At this moment the most probable reason is that you are not compiling the mmcv kernels for compute capability 8_6 for windows wheel. As a result, mmcv does not work on RTX3XXX devices on Windows.

[fcakyon]

I found similar errors on mmcv+windows+compute capability 8_6 gpus: MMCV(1.4.1) + Windows + RTX3070: https://github.com/open-mmlab/mmtracking/issues/378 MMCV(1.4.0) + Windows + RTX3080: https://github.com/open-mmlab/mmcv/issues/1559 MMCV(1.4.0) + Windows + RTX3080: https://github.com/open-mmlab/mmaction2/issues/1308

[fcakyon]

Btw this error is not present with mmcv 1.4.2, torch 1.9.0 and cuda 11.1:

conda create -n mmlab python=3.9 -y 
conda activate mmlab 

conda install pytorch=1.9.0 torchvision=0.10.0 cudatoolkit=11.1 -c pytorch -c conda-forge -y
pip install mmcv-full==1.4.2 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html

pip install git+https://github.com/open-mmlab/mmdetection.git@master

[zhouzaida]

Yet, thank you very much for your feedback! We checked our logs and found we missed the compute capability 8_6. We will re-compile and upload the new packages in the next few days.

[zhouzaida]

Btw this error is not present with mmcv 1.4.2, torch 1.9.0 and cuda 11.1:

conda create -n mmlab python=3.9 -y 
conda activate mmlab 

conda install pytorch=1.9.0 torchvision=0.10.0 cudatoolkit=11.1 -c pytorch -c conda-forge -y
pip install mmcv-full==1.4.2 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html

pip install git+https://github.com/open-mmlab/mmdetection.git@master

Hi, we have updated the pre-built packages for mmcv-full 1.4.1 and 1.4.2. Please have a try.

[fcakyon]

@zhouzaida thanks a lot for your fast responses, i confirmed this working:

• setup env:

conda create -n mmlab python=3.9 -y 
conda activate mmlab 

conda install pytorch=1.10.0 torchvision=0.11.1 cudatoolkit=11.3 -c pytorch -y
pip install mmcv-full==1.4.2 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html

pip install mmdet==2.20.0

• confirm cuda and pytorch installation:

# Check Pytorch installation
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())

# Check MMDetection installation
import mmdet
print(mmdet.__version__)

# Check mmcv installation
from mmcv.ops import get_compiling_cuda_version, get_compiler_version
print(get_compiling_cuda_version())
print(get_compiler_version())

1.10.0 True
2.20.0
11.3
MSVC 192930137

• confirm mmcv installation:

python -c "from mmcv.ops import RoIPool"

no error

• clone mmdet repo for config files:

git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection

• download mmdet weight:

import urllib.request
from pathlib import Path

def download_from_url(from_url: str, to_path: str):
    Path(to_path).parent.mkdir(parents=True, exist_ok=True)
    urllib.request.urlretrieve(
        from_url,
        to_path,
    )

from_url='https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_tiny_8x8_300e_coco/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth'
to_path='weight.pth'
download_from_url(from_url, to_path)

• attempt to perform inference:

from mmdet.apis import inference_detector, init_detector

# Choose to use a config and initialize the detector
config = 'configs/yolox/yolox_tiny_8x8_300e_coco.py'
# Setup a checkpoint file to load
checkpoint = 'weight.pth'
# initialize the detector
model = init_detector(config, checkpoint, device='cuda:0')
# Use the detector to do inference
img = 'demo/demo.jpg'
result = inference_detector(model, img)

no error :rocket: