search

open-mmlab / mmpose

OpenMMLab Pose Estimation Toolbox and Benchmark.
https://mmpose.readthedocs.io/en/latest/
Apache License 2.0
5.82k stars 1.24k forks source link

when deploy mmpose rtmpose, for example, pt to onnx. It can not detect the pose since it locks the detection model. #2109

Closed lelechen63 closed 1 year ago

lelechen63 commented 1 year ago

I follow the instruction. And use the following command to deploy the model. It can output onnx model. However, there are no pose detection results on images since you did not have the port for the detection model. I would suggest you include a new deploy function that incorporates the detection model in the pytorch2onnx. python tools/deploy.py \ configs/mmpose/pose-detection_simcc_onnxruntime_dynamic.py \ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py\ ../codebase/models/rtmpose/rtmpose/body_2d_keypoint/rtmpose-tiny_simcc-aic-coco_pt-aic-coco_420e-256x192-cfc8f33d_20230126.pth \ ./25742375907_cam3.png \ --work-dir ./rtm_onnx \ --device cpu \ --show

Tau-J commented 1 year ago

Thanks for using MMPose. As a script for exporting the model, we think it is not necessary to add the detection method in deploy.py. We provide a human-pose.jpg under mmdeploy/demo/resources/ for verifying whether the model is converted successfully, and you can also quickly get a validation image through screenshots and cropping.

lelechen63 commented 1 year ago

Thanks for using MMPose. As a script for exporting the model, we think it is not necessary to add the detection method in deploy.py. We provide a human-pose.jpg under mmdeploy/demo/resources/ for verifying whether the model is converted successfully, and you can also quickly get a validation image through screenshots and cropping.

Oh yea, that make sense. Thanks. One more question, the rtmdetection model contains hard sigmoid which is not supported by on onnx2dlc. Will you change the hard sigmoid to sigmoid function for snpe?

Tau-J commented 1 year ago

You can discuss issues related to RTMDet with the MMDetection repo team, but MMPose does not involve the development of RTMDet.

Serdnad commented 1 year ago

I had a very similar question to lelechen's original, about what the currently recommended way is to combine ONNX/TensorRT versions of mmdet models with mmpose models. The new Inferencer class is great for prototyping and evaluating different models, but the few examples I've found of exporting mmpose models using mmdeploy don't seem to include a bounding box step. And like lelechen also mentioned, that step is crucial to the accuracy of a lot of these models.

Tau-J commented 1 year ago

Hi @Serdnad , for top-down algorithms like RTMPose, it is assumed that the user already has an independent detector to predict bounding boxes. Therefore, the exported mmpose model does not include the step of generating bounding boxes. In the RTMPose documentation, we provide scipts for exporting detection models (such as RTMDet) and offer det_pose and pose_tracker for joint inference.