search

tijiang13 / InstantAvatar

InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds (CVPR 2023)
376 stars 32 forks source link

SMPL-X support #45

Closed KosukeFukazawa closed 1 year ago

KosukeFukazawa commented 1 year ago

Hello, thank you for sharing your work! I would like to use SMPL-X instead of SMPL for detailed representation of faces and hands. I have already finished parameter estimation for SMPL-X. Is it possible to support SMPL-X with FastSNARF deformation? If so, how should I change the code?

tijiang13 commented 1 year ago

Hi Kosuke,

It's possible. You can refer to https://github.com/Skype-line/X-Avatar (where we basically do that for scans). But I guess it might be tricky for videos due to the occlusion.

Best, Tianjian

KosukeFukazawa commented 1 year ago

Thank you for your reply. I would like to use multi-view videos to avoid occlusion. Is it possible to extend to multiple-view setting by simply changing the input on InstantAvatar?

tijiang13 commented 1 year ago

Hi Kosuke,

To me the main issue with fingers lies in self-occlusion, and I'm not entirely sure that using multi-view inputs will bring about a significant difference.

As for your question, adapting it for multi-view inputs should be relatively straightforward. But you'll need to make modifications to the dataloader, as it currently assumes that all frames share the same intrinsic matrices. See https://github.com/tijiang13/InstantAvatar/blob/master/instant_avatar/datasets/custom.py#L42

Best, Tianjian

KosukeFukazawa commented 1 year ago

Thank you very much! I'll try it.