Jianfeng777
commented
1 year ago 虽然成功转换了一个onnx出来，但是用它来推理我的照片也是没有结果的，显示如下。 (mmdeploy-gpu) jianfeng@Administrator:/mnt/e/AI/mmdeploy$ python /mnt/e/AI/mmdeploy/demo/python/object_detection.py cuda /mnt/e/AI/mmdeploy/output/test /mnt/e/AI/mmdeploy/project/test/demo.jpg loading libmmdeploy_trt_net.so ... failed to load library libmmdeploy_trt_net.so loading libmmdeploy_ort_net.so ... [2023-09-14 10:12:53.911] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "/mnt/e/AI/mmdeploy/output/test" Total time to process the directory: 0.0025 seconds
irexyc
github-actions[bot]
happybear1015
Checklist
Describe the bug
在模型转换中遇到的 `(mmdeploy) jianfeng@Administrator:/mnt/e/AI/mmdeploy$ python tools/deploy.py /mnt/e/AI/mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py /mnt/e/AI/mmdeploy/project/test/rtmdet_tiny_8xb32-300e_coco.py /mnt/e/AI/mmdeploy/project/test/rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth /mnt/e/AI/mmdeploy/project/test/demo.jpg --work-dir output/test --device cuda --dump-info 09/14 10:07:59 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:07:59 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:08:01 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess 09/14 10:08:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:08:01 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. Loads checkpoint by local backend from path: /mnt/e/AI/mmdeploy/project/test/rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth The model and loaded state dict do not match exactly
unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std
09/14 10:08:03 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 09/14 10:08:03 - mmengine - INFO - Export PyTorch model to ONNX: output/test/end2end.onnx. /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! ys_shape = tuple(int(s) for s in ys.shape) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/mmcv/ops/nms.py:285: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/mmcv/ops/nms.py:286: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. score_threshold = torch.tensor([score_threshold], dtype=torch.float32) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. k = torch.tensor(k, device=input.device, dtype=torch.long) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/mmcv/ops/nms.py:44: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! score_threshold = float(score_threshold) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/mmcv/ops/nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! iou_threshold = float(iou_threshold) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert boxes.size(1) == 4 /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert boxes.size(0) == scores.size(0) /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:5589: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results. warnings.warn( 09/14 10:08:06 - mmengine - INFO - Execute onnx optimize passes. 09/14 10:08:06 - mmengine - WARNING - Can not optimize model, please build torchscipt extension. More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md ============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
09/14 10:08:06 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx 09/14 10:08:07 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process 09/14 10:08:07 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend 09/14 10:08:07 - mmengine - INFO - visualize onnxruntime model start. 09/14 10:08:09 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:08:09 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:08:09 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 09/14 10:08:09 - mmengine - INFO - Successfully loaded onnxruntime custom ops from /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/lib/libmmdeploy_onnxruntime_ops.so /home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider' warnings.warn( 2023-09-14:10:08:10 - root - ERROR - Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0] Traceback (most recent call last): File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/utils/utils.py", line 41, in target_wrapper result = target(args, kwargs) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/apis/visualize.py", line 72, in visualize_model result = model.test_step(model_inputs)[0] File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step return self._run_forward(data, mode='predict') # type: ignore File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 340, in _run_forward results = self(data, mode=mode) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, *kwargs) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 192, in forward outputs = self.predict(inputs) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 292, in predict outputs = self.wrapper({self.input_name: imgs}) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/backend/onnxruntime/wrapper.py", line 84, in forward self.io_binding.bind_input( File "/home/jianfeng/miniconda3/envs/mmdeploy/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 496, in bind_input self._iobinding.bind_input( RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0] 09/14 10:08:10 - mmengine - ERROR - tools/deploy.py - create_process - 82 - visualize onnxruntime model failed.`
Reproduction
就是tools里的deploy.py
Environment
Error traceback