search

davrempe / humor

Code for ICCV 2021 paper "HuMoR: 3D Human Motion Model for Robust Pose Estimation"
MIT License
522 stars 72 forks source link

Is it possible to use other 2d keypoint detection methods? #26

Closed atodniAr closed 2 years ago

atodniAr commented 2 years ago

I'm wondering if it's possible to use other pretrained 17 points (COCO format) 2D detection methods instead?

davrempe commented 2 years ago

This is not currently supported, but should be relatively straightforward to add if you want to implement it.

The main thing that needs to be changed is the smpl_to_openpose function which is called when creating the fitting loss to give correspondences between the 3D joints from the SMPL model and the 2D detected joints being used. Right now this function supports COCO-25 and COCO-19 (although I've only tested COCO-19), but it can be easily extended to other COCO formats. You will also need to update OP_IGNORE_JOINTS and OP_NUM_JOINTS which defines which joint indices to not use for the fitting loss.

By default, run_fitting.py calls OpenPose with the COCO-25 format, but you can use the --op-keypts argument to pass in your own OpenPose detections with a different skeleton as described in the README. Or you can change the OpenPose command here to use a different skeleton.