[Bug] RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]

Checklist

[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[X] 3. The bug has not been fixed in the latest version.

Describe the bug

add --device cuda:0 failed, and remove can success

cd /project/mmdeploy

python tools/deploy.py \
    configs/mmdet/detection/detection_onnxruntime_dynamic.py \
    /project/mmdetection/configs/faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py \
    /project/mmdetection/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    /project/mmdeploy/data/coco/val2017/000000000110.jpg \
    --work-dir /project/mmdeploy/work_dir_faster_rcnn \
    --device cuda:0 \
    --dump-info

root@project-10489-instance-20306-ov-ide-674f4b55ff-2bjl6:~/mmdeploy# python tools/deploy.py \
> configs/mmdet/detection/detection_onnxruntime_dynamic.py \
> /project/mmdetection/configs/faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py \
> /project/mmdetection/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
> /project/mmdeploy/data/coco/val2017/000000000110.jpg \
> --work-dir /project/mmdeploy/work_dir_faster_rcnn \
> --device cuda:0 \
> --dump-info
[2022-11-02 12:14:25.967] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
[2022-11-02 12:14:28.315] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:29 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
[2022-11-02 12:14:31.744] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
11/02 12:14:31 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
11/02 12:14:32 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:32 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:14:32 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
local loads checkpoint from path: /project/mmdetection/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
11/02 12:14:40 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
11/02 12:14:40 - mmengine - INFO - Export PyTorch model to ONNX: /project/mmdeploy/work_dir_faster_rcnn/end2end.onnx.
/project/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/usr/local/lib/python3.7/dist-packages/mmdet/models/dense_heads/anchor_head.py:115: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/usr/local/lib/python3.7/dist-packages/mmdet/models/task_modules/prior_generators/anchor_generator.py:341: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` 
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/usr/local/lib/python3.7/dist-packages/mmdet/models/task_modules/prior_generators/anchor_generator.py:378: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` 
  '``single_level_grid_anchors`` would be deprecated soon. '
/project/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py:88: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/project/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
/project/mmdeploy/mmdeploy/pytorch/functions/topk.py:33: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  return ctx.origin_func(input, k, dim=dim, largest=largest, sorted=sorted)
/project/mmdeploy/mmdeploy/codebase/mmdet/models/task_modules/coders/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(0) == bboxes.size(0)
/project/mmdeploy/mmdeploy/codebase/mmdet/models/task_modules/coders/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(1) == bboxes.size(1)
/project/mmdeploy/mmdeploy/codebase/mmdet/deploy/utils.py:47: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  assert len(max_shape) == 2, '`max_shape` should be [h, w]'
/project/mmdeploy/mmdeploy/codebase/mmdet/models/layers/bbox_nms.py:93: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/project/mmdeploy/mmdeploy/codebase/mmdet/models/layers/bbox_nms.py:94: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/project/mmdeploy/mmdeploy/mmcv/ops/nms.py:40: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  score_threshold = float(score_threshold)
/project/mmdeploy/mmdeploy/mmcv/ops/nms.py:41: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  iou_threshold = float(iou_threshold)
/usr/local/lib/python3.7/dist-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/usr/local/lib/python3.7/dist-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/usr/local/lib/python3.7/dist-packages/mmcv/ops/nms.py:30: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if max_num > 0:
/usr/local/lib/python3.7/dist-packages/mmcv/ops/roi_align.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert rois.size(1) == 5, 'RoI must be (idx, x1, y1, x2, y2)!'
/usr/local/lib/python3.7/dist-packages/torch/onnx/symbolic_opset9.py:2819: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")
11/02 12:14:54 - mmengine - INFO - Execute onnx optimize passes.
11/02 12:14:57 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
11/02 12:14:58 - mmengine - INFO - visualize onnxruntime model start.
[2022-11-02 12:15:00.411] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
11/02 12:15:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:15:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:15:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:15:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
11/02 12:15:01 - mmengine - INFO - Successfully loaded onnxruntime custom ops from             /project/mmdeploy/mmdeploy/lib/libmmdeploy_onnxruntime_ops.so
/usr/local/lib/python3.7/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
  "Available providers: '{}'".format(name, ", ".join(available_provider_names)))
2022-11-02 12:15:01.939746145 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2612'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939781858 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2610'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939793056 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2609'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939802135 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2608'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939810609 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2606'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939819083 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2604'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939828465 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2602'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939837242 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2601'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939853282 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2600'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939861454 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2599'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939871441 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2597'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939880218 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2595'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939889600 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2593'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939899890 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2592'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939916233 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2591'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939925615 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2590'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939938024 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2579'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939947103 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2576'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939957090 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2572'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939968288 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2570'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939979486 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2569'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.939993105 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2568'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940006422 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2566'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940018225 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2565'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940027305 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2564'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940049701 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2603'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940067557 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2558'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940079965 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2557'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940091769 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2556'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940101756 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2554'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940111138 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2552'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940138074 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2551'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940152601 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2550'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940202840 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2549'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940215854 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2547'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940225841 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2567'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940238552 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2546'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940248540 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2596'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940261856 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2545'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940270936 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2542'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940280620 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2538'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940292726 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2536'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940302714 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2535'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940311793 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2534'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940324504 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2574'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940334492 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2526'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940345084 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2553'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940356888 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2589'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940366572 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2611'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940379283 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2523'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940391995 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2541'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940406824 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2539'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940418022 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2524'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940430128 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2607'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940441629 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2587'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940452222 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2544'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940467051 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2509'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940476433 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2613'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940488539 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2508'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940499132 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2518'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940509725 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2540'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940522738 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2559'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940532423 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2582'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940543016 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2573'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940556030 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2555'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940565714 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2530'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940575702 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2543'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940588715 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2515'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940598703 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2548'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940610809 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2571'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940620796 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2605'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940631086 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2575'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940645311 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2588'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940654995 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2505'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940665891 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2513'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940680418 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2528'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940692221 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2522'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940703116 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2584'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940726117 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2506'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940735499 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2510'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940747303 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2598'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940767883 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2519'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940782712 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2511'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940799358 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2577'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940811464 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2512'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940821149 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2531'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940831741 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2514'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940841123 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2516'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940854440 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2517'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940866243 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2520'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940900442 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2521'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940915877 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2525'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940930102 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2527'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940944023 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2529'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940961577 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2594'. It is not used by any node and should be removed from the model.
2022-11-02 12:15:01.940971564 [W:onnxruntime:, graph.cc:3211 CleanUnusedInitializers] Removing initializer '2532'. It is not used by any node and should be removed from the model.
2022-11-02:12:15:09 - root - ERROR - Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
Traceback (most recent call last):
  File "/project/mmdeploy/mmdeploy/utils/utils.py", line 41, in target_wrapper
    result = target(*args, **kwargs)
  File "/project/mmdeploy/mmdeploy/apis/visualize.py", line 72, in visualize_model
    result = model.test_step(model_inputs)[0]
  File "/usr/local/lib/python3.7/dist-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
    return self._run_forward(data, mode='predict')  # type: ignore
  File "/usr/local/lib/python3.7/dist-packages/mmengine/model/base_model/base_model.py", line 301, in _run_forward
    results = self(**data, mode=mode)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/project/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 176, in forward
    outputs = self.predict(inputs)
  File "/project/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 258, in predict
    outputs = self.wrapper({self.input_name: imgs})
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/project/mmdeploy/mmdeploy/backend/onnxruntime/wrapper.py", line 90, in forward
    buffer_ptr=input_tensor.data_ptr())
  File "/usr/local/lib/python3.7/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 384, in bind_input
    element_type, shape, buffer_ptr)
RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
11/02 12:15:10 - mmengine - ERROR - tools/deploy.py - create_process - 78 - visualize onnxruntime model failed.

Reproduction

add --device cuda:0 failed,remove success

cd /project/mmdeploy

python tools/deploy.py \
    configs/mmdet/detection/detection_onnxruntime_dynamic.py \
    /project/mmdetection/configs/faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py \
    /project/mmdetection/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    /project/mmdeploy/data/coco/val2017/000000000110.jpg \
    --work-dir /project/mmdeploy/work_dir_faster_rcnn \
    --device cuda:0 \
    --dump-info

Environment

root@project-10489-instance-20306-ov-ide-674f4b55ff-2bjl6:~/mmdeploy# python tools/check_env.py 
11/02 12:10:58 - mmengine - INFO - 

11/02 12:10:58 - mmengine - INFO - **********Environmental information**********
11/02 12:10:58 - mmengine - INFO - sys.platform: linux
11/02 12:10:58 - mmengine - INFO - Python: 3.7.5 (default, Dec  9 2021, 17:04:37) [GCC 8.4.0]
11/02 12:10:58 - mmengine - INFO - CUDA available: True
11/02 12:10:58 - mmengine - INFO - numpy_random_seed: 2147483648
11/02 12:10:58 - mmengine - INFO - GPU 0: GeForce RTX 2080 Ti
11/02 12:10:58 - mmengine - INFO - CUDA_HOME: /usr/local/cuda
11/02 12:10:58 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.1, V11.1.105
11/02 12:10:58 - mmengine - INFO - GCC: x86_64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
11/02 12:10:58 - mmengine - INFO - PyTorch: 1.10.0+cu111
11/02 12:10:58 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

11/02 12:10:58 - mmengine - INFO - TorchVision: 0.11.0+cu111
11/02 12:10:58 - mmengine - INFO - OpenCV: 4.5.5
11/02 12:10:58 - mmengine - INFO - MMEngine: 0.2.0
11/02 12:10:58 - mmengine - INFO - MMCV: 2.0.0rc2
11/02 12:10:58 - mmengine - INFO - MMCV Compiler: GCC 7.3
11/02 12:10:58 - mmengine - INFO - MMCV CUDA Compiler: 11.1
11/02 12:10:58 - mmengine - INFO - MMDeploy: 0.9.0+185d838
11/02 12:10:58 - mmengine - INFO - 

11/02 12:10:58 - mmengine - INFO - **********Backend information**********
11/02 12:10:59 - mmengine - INFO - onnxruntime: 1.8.1   ops_is_avaliable : True
11/02 12:10:59 - mmengine - INFO - tensorrt: None       ops_is_avaliable : False
11/02 12:10:59 - mmengine - INFO - ncnn: None   ops_is_avaliable : False
11/02 12:10:59 - mmengine - INFO - pplnn_is_avaliable: False
11/02 12:10:59 - mmengine - INFO - openvino_is_avaliable: True
11/02 12:10:59 - mmengine - INFO - snpe_is_available: False
11/02 12:10:59 - mmengine - INFO - ascend_is_available: False
11/02 12:10:59 - mmengine - INFO - coreml_is_available: False
11/02 12:10:59 - mmengine - INFO - 

11/02 12:10:59 - mmengine - INFO - **********Codebase information**********
11/02 12:10:59 - mmengine - INFO - mmdet:       3.0.0rc2
11/02 12:10:59 - mmengine - INFO - mmseg:       None
11/02 12:10:59 - mmengine - INFO - mmcls:       None
11/02 12:10:59 - mmengine - INFO - mmocr:       None
11/02 12:10:59 - mmengine - INFO - mmedit:      None
11/02 12:10:59 - mmengine - INFO - mmdet3d:     None
11/02 12:10:59 - mmengine - INFO - mmpose:      None
11/02 12:10:59 - mmengine - INFO - mmrotate:    None
11/02 12:10:59 - mmengine - INFO - mmaction:    None

```shell
root@project-10489-instance-20306-ov-ide-674f4b55ff-2bjl6:~/mmdeploy# echo $PATH
/opt/intel/openvino/deployment_tools/model_optimizer:/usr/local/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

root@project-10489-instance-20306-ov-ide-674f4b55ff-2bjl6:~/mmdeploy# echo $LD_LIBRARY_PATH
/opt/intel/openvino/deployment_tools/ngraph/lib:/opt/intel/opencl:/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/gna/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/mkltiny_lnx/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/tbb/lib:/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64:/usr/local/lib:/usr/local/cuda/lib64/:/usr/local/nvidia/lib:/usr/local/nvidia/lib64

root@project-10489-instance-20306-ov-ide-674f4b55ff-2bjl6:~/mmdeploy# echo $PYTHONPATH
/usr/local/lib/python3.7/dist-packages/:/opt/intel/openvino_2021.3.394/python/python3.7:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/accuracy_checker:/opt/intel/openvino/deployment_tools/model_optimizer:



### Error traceback

_No response_

open-mmlab / mmdeploy

[Bug] RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0] #1285

Checklist

Describe the bug

Reproduction

Environment