search

david-svitov / HAHA

HAHA: Highly Articulated Gaussian Human Avatars with Textured Mesh Prior
89 stars 9 forks source link

Refine SMPL-X #8

Open Helen-liang opened 2 months ago

Helen-liang commented 2 months ago

Very nice work! In your repo, you mentioned that "finetuned InstantAvatar pre-processed poses to get SMPL-X fits.", could you release the finetune code? How to refine the SMPL-X parameters?

david-svitov commented 2 months ago

Hi! We refined SMPL-X as follows:

  1. First we converted SMPL to SMPL-X using this code: https://github.com/vchoutas/smplx/blob/main/config_files/smpl2smplx.yaml
  2. We then used SMPLify-x to refine the SMPL-X parameters: https://github.com/vchoutas/smplify-x

To do this, we froze all the joints except the hand joints and finetuned them.

haoz19 commented 1 month ago

Hi,

Thanks for the instruction!

I understand we should follow 

python -m transfer_model --exp-cfg config_files/smpl2smplx.yaml

to convert SMPL to SMPL-X.

While I'm so sorry, I'm still a little confused about: How to refine the SMPL-X parameters. Why not directly use 

python smplifyx/main.py --config cfg_files/fit_smplx.yaml 
    --data_folder DATA_FOLDER 
    --output_folder OUTPUT_FOLDER 
    --visualize="True/False"
    --model_folder MODEL_FOLDER
    --vposer_ckpt VPOSER_FOLDER
    --part_segm_fn smplx_parts_segm.pkl

to fix the image with the SMPLify-x model?

Thanks a lot for your instruction!

david-svitov commented 1 month ago

We used pre-trained SMPL models from the dataset. Therefore, we first converted them into SMPL-X, and then finetuned only the position of the hands. So that the rest of the body matches the original SMPL.

If your task does not need to use existing SMPLs, then it is better to just train SMPL-X as you suggest.