[Bug] When I try to convert rtmpose from onnx to ncnn, I get an error

da13132 commented 1 year ago

Checklist

[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[X] 3. The bug has not been fixed in the latest version.

Describe the bug

When I try to convert rtmpose from onnx format to ncnn, an error is reported. Suggested useless limit, in fact I can run other translations, only available onnx2ncnn Hope to get your help!

Reproduction

python tools/deploy.py /home/keli/下载/mmdeploy/configs/mmpose/pose-detection_simcc_ncnn_static-256x192.py /home/keli/下载/pose/configs/body_2d_keypoint/rtmpose/coco/rtmpose-s_8xb256-420e_coco-256x192.py /home/keli/下载/pose/pthfile/rtmpose/rtmpose-s_simcc-coco_pt-aic-coco_420e-256x192-8edcf0d7_20230127.pth demo/resources/human-pose.jpg --work-dir rtmpose_small_ncnn --device cpu --show --dump-info

Environment

08/08 16:41:21 - mmengine - INFO - 

08/08 16:41:21 - mmengine - INFO - **********Environmental information**********
08/08 16:41:21 - mmengine - INFO - sys.platform: linux
08/08 16:41:21 - mmengine - INFO - Python: 3.9.17 (main, Jul  5 2023, 20:41:20) [GCC 11.2.0]
08/08 16:41:21 - mmengine - INFO - CUDA available: True
08/08 16:41:21 - mmengine - INFO - numpy_random_seed: 2147483648
08/08 16:41:21 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3090
08/08 16:41:21 - mmengine - INFO - CUDA_HOME: /usr/local/cuda-11.6
08/08 16:41:21 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
08/08 16:41:21 - mmengine - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
08/08 16:41:21 - mmengine - INFO - PyTorch: 1.13.1+cu116
08/08 16:41:21 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.6
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.5  (built against CUDA 11.7)
    - Built with CuDNN 8.3.2
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

08/08 16:41:21 - mmengine - INFO - TorchVision: 0.14.1+cu116
08/08 16:41:21 - mmengine - INFO - OpenCV: 4.8.0
08/08 16:41:21 - mmengine - INFO - MMEngine: 0.8.2
08/08 16:41:21 - mmengine - INFO - MMCV: 2.0.1
08/08 16:41:21 - mmengine - INFO - MMCV Compiler: GCC 9.3
08/08 16:41:21 - mmengine - INFO - MMCV CUDA Compiler: 11.6
08/08 16:41:21 - mmengine - INFO - MMDeploy: 1.2.0+
08/08 16:41:21 - mmengine - INFO - 

08/08 16:41:21 - mmengine - INFO - **********Backend information**********
08/08 16:41:21 - mmengine - INFO - tensorrt:    8.5.3.1
08/08 16:41:21 - mmengine - INFO - tensorrt custom ops: Available
08/08 16:41:21 - mmengine - INFO - ONNXRuntime: 1.8.1
08/08 16:41:21 - mmengine - INFO - ONNXRuntime-gpu:     None
08/08 16:41:21 - mmengine - INFO - ONNXRuntime custom ops:      Available
08/08 16:41:21 - mmengine - INFO - pplnn:       None
08/08 16:41:21 - mmengine - INFO - ncnn:        1.0.20230808
08/08 16:41:21 - mmengine - INFO - ncnn custom ops:     NotAvailable
08/08 16:41:21 - mmengine - INFO - snpe:        None
08/08 16:41:21 - mmengine - INFO - openvino:    None
08/08 16:41:21 - mmengine - INFO - torchscript: 1.13.1+cu116
08/08 16:41:21 - mmengine - INFO - torchscript custom ops:      NotAvailable
08/08 16:41:21 - mmengine - INFO - rknn-toolkit:        None
08/08 16:41:21 - mmengine - INFO - rknn-toolkit2:       None
08/08 16:41:21 - mmengine - INFO - ascend:      None
08/08 16:41:21 - mmengine - INFO - coreml:      None
08/08 16:41:21 - mmengine - INFO - tvm: None
08/08 16:41:21 - mmengine - INFO - vacc:        None
08/08 16:41:21 - mmengine - INFO - 

08/08 16:41:21 - mmengine - INFO - **********Codebase information**********
08/08 16:41:22 - mmengine - INFO - mmdet:       3.0.0
08/08 16:41:22 - mmengine - INFO - mmseg:       None
08/08 16:41:22 - mmengine - INFO - mmpretrain:  1.0.0rc8
08/08 16:41:22 - mmengine - INFO - mmocr:       None
08/08 16:41:22 - mmengine - INFO - mmagic:      None
08/08 16:41:22 - mmengine - INFO - mmdet3d:     None
08/08 16:41:22 - mmengine - INFO - mmpose:      1.0.0
08/08 16:41:22 - mmengine - INFO - mmrotate:    None
08/08 16:41:22 - mmengine - INFO - mmaction:    None
08/08 16:41:22 - mmengine - INFO - mmrazor:     None
08/08 16:41:22 - mmengine - INFO - mmyolo:      0.6.0

Error traceback

08/08 16:50:59 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/08 16:50:59 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/08 16:51:00 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
08/08 16:51:01 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/08 16:51:01 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: /home/keli/下载/pose/pthfile/rtmpose/rtmpose-s_simcc-coco_pt-aic-coco_420e-256x192-8edcf0d7_20230127.pth
/home/keli/下载/pose/mmpose/datasets/datasets/utils.py:102: UserWarning: The metainfo config file "configs/_base_/datasets/coco.py" does not exist. A matched config file "/home/keli/下载/pose/mmpose/.mim/configs/_base_/datasets/coco.py" will be usestead.
  warnings.warn(
08/08 16:51:01 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
08/08 16:51:01 - mmengine - INFO - Export PyTorch model to ONNX: rtmpose_small_ncnn/end2end.onnx.
08/08 16:51:01 - mmengine - WARNING - Can not find models.yolox_pose_head.YOLOXPoseHead.predict, function rewrite will not be applied
08/08 16:51:01 - mmengine - WARNING - Can not find models.yolox_pose_head.YOLOXPoseHead.predict_by_feat, function rewrite will not be applied
08/08 16:51:01 - mmengine - WARNING - Can not find mmdet.models.dense_heads.RPNHead.get_bboxes, function rewrite will not be applied
/home/keli/下载/mmdeploy/mmdeploy/pytorch/functions/size.py:23: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constanin the future. This means that the trace might not generalize to other inputs!
  ret = [int(r) for r in ret]
/home/keli/下载/mmdeploy/mmdeploy/pytorch/functions/getattribute.py:19: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as aonstant in the future. This means that the trace might not generalize to other inputs!
  ret = torch.Size([int(s) for s in ret])
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: The shape inference of mmdeploy::AdaptiveAvgPool2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/keli/下载/mmdeploy/mmdeploy/pytorch/functions/linear.py:27: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please on a GitHub issue if you need this functionality..
  return g.op(*args, alpha_f=1.0, beta_f=1.0, transA_i=0, transB_i=1)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:687: UserWarning: The shape inference of mmdeploy::AdaptiveAvgPool2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:1178: UserWarning: The shape inference of mmdeploy::AdaptiveAvgPool2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
08/08 16:51:09 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
08/08 16:51:10 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
Traceback (most recent call last):
  File "/home/keli/下载/mmdeploy/tools/deploy.py", line 335, in <module>
    main()
  File "/home/keli/下载/mmdeploy/tools/deploy.py", line 259, in main
    backend_files = to_backend(
  File "/home/keli/下载/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/home/keli/下载/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/home/keli/下载/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/home/keli/下载/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/keli/下载/mmdeploy/mmdeploy/apis/utils/utils.py", line 98, in to_backend
    return backend_mgr.to_backend(
  File "/home/keli/下载/mmdeploy/mmdeploy/backend/ncnn/backend_manager.py", line 141, in to_backend
    from_onnx(onnx_path, osp.join(work_dir, onnx_name))
  File "/home/keli/下载/mmdeploy/mmdeploy/backend/ncnn/onnx2ncnn.py", line 71, in from_onnx
    ret_code = call([onnx2ncnn_path, onnx_path, save_param, save_bin])
  File "/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/subprocess.py", line 349, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/subprocess.py", line 1837, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: ''

da13132 commented 1 year ago

The preliminary judgment is that the mmdeploy/mmdeploy/backend/ncnn/onnx2ncnn.py return result of ret_code = call([onnx2ncnn_path, onnx_path, save_param, save_bin]) is empty

RunningLeon commented 1 year ago

suggest that the whole path of mmdeploy does not have Chinese characters.

/home/keli/下载/mmdeploy

irexyc commented 1 year ago

mmdeploy 你应该不是自己编译的吧?

转ncnn依赖mmdeploy_onnx2ncnn这个bin文件，需要编译mmdeploy的时候打开ncnn的后端。

可以使用这个平台在线转：https://platform.openmmlab.com/deploee

da13132 commented 1 year ago

mmdeploy 你应该不是自己编译的吧?

转ncnn依赖mmdeploy_onnx2ncnn这个bin文件，需要编译mmdeploy的时候打开ncnn的后端。

可以使用这个平台在线转：https://platform.openmmlab.com/deploee

Thanks for your help, after I compiled the ncnn custom operator, the conversion was successful

da13132 commented 1 year ago

suggest that the whole path of mmdeploy does not have Chinese characters.

/home/keli/下载/mmdeploy

Thank you， i will keep an eye out.

github-actions[bot] commented 1 year ago

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

github-actions[bot] commented 1 year ago

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

fire717 commented 10 months ago

mmdeploy 你应该不是自己编译的吧? 转ncnn依赖mmdeploy_onnx2ncnn这个bin文件，需要编译mmdeploy的时候打开ncnn的后端。可以使用这个平台在线转：https://platform.openmmlab.com/deploee

Thanks for your help, after I compiled the ncnn custom operator, the conversion was successful

So how did you compile the ncnn custom operator? Did you compile the ncnn custom operator because you use your own model? I wanna convert the official original rtmpose model, and meet same error.

and all my path is in English, also have runpython3 tools/scripts/build_ubuntu_x64_ncnn.py $(nproc)

open-mmlab / mmdeploy