search

open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.77k stars 636 forks source link

Heatmap to keypoint function from ONNX model #872

Open Monalsingh opened 2 years ago

Monalsingh commented 2 years ago

Hello,

I have converted a MMPose model into onnx using the below command:

python /home/monal/mmdeploy/tools/deploy.py \
    ${MMDEPLOY_DIR}/configs/mmpose/pose-detection_onnxruntime_static.py \
    /home/monal/mmpose/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/mobilenetv2_coco_256x192.py \
    ${CHECKPOINT_DIR}/mobilenetv2_coco_256x192-d1e58e7b_20200727.pth \
    /home/monal/mmdeploy/demo/resources/human-pose.jpg \
    --work-dir ${WORK_DIR} \
    --device cuda:0

Now the thing is I am serving the model in Triton interface so I need to inference the model using the Triton way. That step is done.

The problem is the output of the ONNX model inference is of size (1, 17, 64, 48). I cannot do inference with 'from mmdeploy_python import PoseDetector'.

Is there a way or function available to convert heatmap to keypoint.

Input: heatmap output: keypoint.

Thanks, Monal

RunningLeon commented 2 years ago

@Monalsingh Hi, if you're using mmdeploy-sdk to do inference, you have to add --dump-info while calling tools/deploy.py and build mmdeploy-sdk from source by following get_started. Besides, you can implement the decoding part yourself by referring to https://github.com/open-mmlab/mmpose/blob/7616b62eabdfed9cbd02fbbfa9411ff37e072be7/mmpose/models/heads/topdown_heatmap_base_head.py#L40.

lvhan028 commented 2 years ago

Hi, @Monalsingh I am afraid that MMDeploy cannot meet your requirement. PoseDetector in mmdeploy_python package doesn't provide API to convert heatmap to keypoint. You can either make mmdeploy as a backend of Triton server or implement keypoint extraction in your client.