yoyo-nb / Thin-Plate-Spline-Motion-Model

[CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.
MIT License
3.44k stars 558 forks source link

Plans to modify it to perform partswap/faceswap? #17

Open instant-high opened 2 years ago

instant-high commented 2 years ago

Thank you for this great repo.

I'm using the original FOMM for about 2 years. But your modification gives much better results. Are you planning to modify it so we can use it for face swap / part swap like the motion co-segmentation branch of FOMM?

I've made major modifications to the motion co-segmentation repo, so that now it performs 'automatic' face tracking / cropping / face swap /colour correction and putting all together again.

It would be nice to do that with a version that gives even better results.

yoyo-nb commented 2 years ago

The current version of the model cannot do part swap tasks

Because the current model learns by unsupervised method, the contribution of each TPS to the optical flow is not concentrated in one region like FOMM, but scattered in various locations on the face. And good motion transfer results can only be obtained when multiple TPSs cooperate.

instant-high commented 2 years ago

Thanks for your reply.