[Bug] rtmdetection and rtmpose onnx to dlc model failed.

[lelechen63]

Checklist

• [X] I have searched related issues but cannot get the expected help.
• [X] 2. I have read the FAQ documentation but cannot get the expected help.
• [X] 3. The bug has not been fixed in the latest version.

Describe the bug

I followed the instruction from https://github.com/open-mmlab/mmpose/tree/1.x/projects/rtmpose. And. I can transfer the .pt model to onnx with following command:

convert the rtmdet_tiny model

python tools/deploy.py \ configs/mmdet/detection/detection_onnxruntime_static.py \ ../codebase/models/rtmdet_tiny_8xb32-300e_coco.py\ ../codebase/models/rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth \ ./25742375907_cam3.png \ --work-dir ./rtm_onnx \ --device cpu \ --show

convert the rtmpose-tiny model

python tools/deploy.py \ configs/mmpose/pose-detection_simcc_onnxruntime_dynamic.py \ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py\ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-tiny_simcc-aic-coco_pt-aic-coco_420e-256x192-cfc8f33d_20230126.pth \ ./25742375907_cam3.png \ --work-dir ./rtm_onnx \ --device cpu \ --show

However, I can not convert them into snpe (dlc) models with your onnx2dlc.py function. Could you look into it?

I have attached my onnx models (step1:rtmdet_tiny, step2: rtmpose-tiny ). step1.onnx.zip

step2.onnx.zip Thanks for your great work! Cheers!

I also tried pth2snpe command: python tools/deploy.py \ configs/mmpose/pose-detection_snpe_static-256x256.py \ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py\ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-tiny_simcc-aic-coco_pt-aic-coco_420e-256x192-cfc8f33d_20230126.pth \ ./25742375907_cam3.png \ --work-dir ./rtm_onnx \ --device cpu \ --show And I also failed at this step.

Reproduction

convert the rtmdet_tiny model

python tools/deploy.py \ configs/mmdet/detection/detection_onnxruntime_static.py \ ../codebase/models/rtmdet_tiny_8xb32-300e_coco.py\ ../codebase/models/rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth \ ./25742375907_cam3.png \ --work-dir ./rtm_onnx \ --device cpu \ --show

convert the rtmpose-tiny model

python tools/deploy.py \ configs/mmpose/pose-detection_simcc_onnxruntime_dynamic.py \ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py\ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-tiny_simcc-aic-coco_pt-aic-coco_420e-256x192-cfc8f33d_20230126.pth \ ./25742375907_cam3.png \ step1.onnx.zip

--work-dir ./rtm_onnx \
--device cpu \
--show

Environment

same environment as required.

Error traceback

spne does not support hardsigmoid in step 1. There could be some other problems after fixing this problem.

[tpoisonooo]

1. Use mmpose stable branch (for example v1.0.0rc1) instead of 1.x
2. Please give the error log, if spne does not support hardsigmoid is all of them
   • you have to rewrite hardsigmoid with simple operation, here is an example to split Gelu for ncnn https://github.com/open-mmlab/mmdeploy/blob/master/mmdeploy/pytorch/ops/layer_norm.py
   • or use other activation operator

[lelechen63]

1. Use mmpose stable branch (for example v1.0.0rc1) instead of 1.x
2. Please give the error log, if spne does not support hardsigmoid is all of them

• you have to rewrite hardsigmoid with simple operation, here is an example to split Gelu for ncnn https://github.com/open-mmlab/mmdeploy/blob/master/mmdeploy/pytorch/ops/layer_norm.py
• or use other activation operator

It seems like you already implement the hard sigmoid at https://github.com/open-mmlab/mmdeploy/blob/master/mmdeploy/pytorch/ops/hardsigmoid.py. How can I get the new onnx model without the hardsigmoid? Do I need to retrain the model without the hardsigmoid activation? Or is there any cheap way to replace it in pth2onnx step? Thanks