search

yohanshin / WHAM

MIT License
646 stars 71 forks source link

Support SMPL-X estimation? #20

Closed michaelliyunhao closed 8 months ago

michaelliyunhao commented 8 months ago

Nice work! I am wondering if the method can supprot reconstructing world 3D SMPL-X motions?

Thank you!

carlosedubarreto commented 8 months ago

Hello @michaelliyunhao , I received an answer that the result will have the SMPL parameters in january, and following these instructions you can convert SMPL to SMPL-X

https://github.com/vchoutas/smplx/tree/main/transfer_model#smpl-to-smpl-x

(I actually didnt try it yet, I just know that it existis)

yohanshin commented 8 months ago

Thanks @carlosedubarreto ! Yes, you can simply convert SMPL to SMPL-X model through the optimization process. This is the simple solution, but this won't allow reconstructing facial expression and hand pose.

michaelliyunhao commented 8 months ago

Thanks @carlosedubarreto ! Yes, you can simply convert SMPL to SMPL-X model through the optimization process. This is the simple solution, but this won't allow reconstructing facial expression and hand pose. Thanks for your quick reply! I think it is a reasonable strategy to converted the final estimated SMPL to SMPL-X without hand pose and expression, but I am wondering if I can resonstruct global whole body motions with hand and face using monocular 3D mesh recovery methods of SMPL-X such as hybrik-x. Since SHAM seems focusing on optimizing the global root translation which is nothing to do with hands and faces, I am wondering if I can just add the estimated SMPL-X hand and face parameters in camera coordinate on the converted SMPL-X parameters obtained from SHAM?

yohanshin commented 8 months ago

@michaelliyunhao

Yes, get global body motion of SMPL-X from WHAM estimation (convert SMPL to SMPL-X) and add hand pose / facial expression from Hybrik-x sounds like a solid way to recover full-body motion. But I guess converting SMPL to SMPL-X runs a bit slowly, won't be real-time in this case.

michaelliyunhao commented 8 months ago

@michaelliyunhao

Yes, get global body motion of SMPL-X from WHAM estimation (convert SMPL to SMPL-X) and add hand pose / facial expression from Hybrik-x sounds like a solid way to recover full-body motion. But I guess converting SMPL to SMPL-X runs a bit slowly, won't be real-time in this case.

Great! Thanks for your reply!