search

jiashunwang / Neural-Pose-Transfer

Neural Pose Transfer by Spatially Adaptive Instance Normalization. In CVPR 2020
Apache License 2.0
146 stars 21 forks source link

Facial expressions and fingers #2

Closed jaygit123 closed 4 years ago

jaygit123 commented 4 years ago

Hello @jiashunwang & team, Great paper. Thanks for sharing the code.

Wish to know if the model will be able to do these:

  1. transfer facial expressions
  2. transfer exact finger positions (all 10 fingers)

Thanks,

walsvid commented 4 years ago

Thank you for your interest, this is a good question. As for your question, I think the network can do this. Since we are focus on the pose transfer task of 3D human mesh, and the SMPL model has no control parameter to generate data for different facial expressions or finger movements, we did not try this experiment. I think using SMPLify-x, a parameterized model which can control expressions and hands or 3DMM for facial mesh to generate training data, our method is possible to transfer fine-grained pose.

jaygit123 commented 4 years ago

Hi @walsvid , thanks for your response. yes, i am aware of SMPL-X and was thinking the same to enable facial expressions and finger movements. have you already tried SMPL-X in your model? if yes, do share code / inputs as to where all we have to change code.

Thanks

walsvid commented 4 years ago

We did some exploration, but we didn’t have thorough experiments on SMPL-X. I think we can not provide the data or code now, this will be a future work.

jaygit123 commented 4 years ago

ok. thanks for the update @walsvid