search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.52k stars 3.22k forks source link

Training for high quality #171

Open randomdev-work opened 4 years ago

randomdev-work commented 4 years ago

Hello, The question was already posted many times about the low resolution output. The only way is to try with higher resolution dataset. Let's say I have a massive amount of people faces in videos (I'm a filmmaker not a stalker).

So how can I create my own? I would love specific steps how to make one.

If my logic is right first I need to process the videos that I have in 1:1 ration and for simplicity 1024x1024 (is big enough I think?) resolution, exporting them either in mp4 or png for quicker process, create a train folder (I imagine it won't take a day to process on my machine...) and then compress it to the tar file or is it creating automatically with the train process? I'm a very beginner in this staff so don't judge me please.

Thanks for taking time to read my stupid request and if it works just fine I can upload it somewhere for everyone.

AliaksandrSiarohin commented 4 years ago

Hi, You need to crop the videos, such that only face is present and everything else is absent (similar to how is crop-video.py works). Tar file is not needed. For quick processing you need a folders with .png images. This would take a lot of space several TB, if the amount of videos is the same as in VoxCeleb. Then you need a machine with many gpus (8 P100 for example), higher resolution requires significantly more resources. Then you can refer to #81 to see how to proceed with training.

randomdev-work commented 4 years ago

Is it possible to cut out most of the training process? Let’s say I will train only on my face and use this later only with video of me or it doesn’t work that way? I had that in mind to record multiple videos with me Standing with a green screen background and from many different angles and movements like intensive blinking or head turning would that work? This way it should analyse me enough I think?

AliaksandrSiarohin commented 4 years ago

If your daset is small you could try this out.

MitalPattani commented 4 years ago

Hello, did you try this, did it improve the results?

randomdev-work commented 4 years ago

Well I didn't get good results with that unfortunately. For my project I settled on upscaling the 256x256 video. It worked for me, because I was using an old photo so I could make it look older so It would look like an old video capture. I only upscaled with my editing software (Adobe After effects), because for my case upscaling AI made my character very sketchy so it wasn't good for me. It isn't the best way we definitely for better use of this model we need to train it on larger dataset. With higher resolution of course.

MitalPattani commented 4 years ago

Awesome thanks!