search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.43k stars 3.2k forks source link

Training with Synthetic data? #315

Open MiloMindbender opened 3 years ago

MiloMindbender commented 3 years ago

I've tried driving some synthetic faces (ie: CG rendered faces) with video from a live person and got pretty good results.

If the goal is to use driving video from a live person to animate a picture of a CG face, what should the model be trained on to get the best results? Videos of humans speaking, videos of CG faces speaking or both?

Attached are a couple of synthetic faces I tried in case anyone wants to experiment. This is pretty good for experimenting as it's easy to change around things like hairstyles and such to see if the animation is effected.

p14 p13 p12

AliaksandrSiarohin commented 3 years ago

If the goal is to use driving video from a live person to animate a picture of a CG face, what should the model be trained on to get the best results? Videos of humans speaking, videos of CG faces speaking or both?

Both, make sure that they are cropped in the same way.