NVlabs / imaginaire

NVIDIA's Deep Imagination Team's PyTorch Library
Other
3.99k stars 444 forks source link

About background movements for talking head generation? #176

Closed daikankan closed 1 year ago

daikankan commented 1 year ago

It seems that we have to fix camera position, while generating training pairs for talkinghead dataset, otherwise It is hard to solve background movements of the generation ? If so, what is the best way for training data pairs generation and augmentation ? If I do scale, shift and rotation augmentation while trainning, driving results are as follows:

https://user-images.githubusercontent.com/20749514/223013634-e31b779e-48cb-4a96-8e1a-1bb830fa4b78.mp4