search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.49k stars 3.21k forks source link

Taichi with my own driver video, help needed #199

Open bigboss97 opened 4 years ago

bigboss97 commented 4 years ago

I tried to follow the instruction. But I failed to get the driver video. So I tried to make my own one. This is the command I've used (probably wrong) 😄 python /bin/first-order-model-master/demo.py --config E:\bin\first-order-model-master\config\taichi-adv-256.yaml --driving_video .\driver.mp4 --source_image .\pose.png --checkpoint E:\bin\first-order-model-master\play\taichi-adv-cpk.pth.tar --adapt_scale

and this is my result hahaha https://www.reddit.com/r/MediaSynthesis/comments/hy6we5/i_think_my_firstordermodel_taichi_has_failed/?utm_source=share&utm_medium=web2x

I've moved very slowly in the video (only little motion blur in my hands) I've used a square video and a square image. The starting poses are very similar. What else do I have to do? Does the driver video need a clear background?

AliaksandrSiarohin commented 4 years ago

Taichi have very poor Generalization, so basically it only works with other Taichi videos.

askbekzod commented 4 years ago

Taichi have very poor Generalization, so basically it only works with other Taichi videos.

Is there any way to train it ourselves, so it would work with our input data?

AliaksandrSiarohin commented 4 years ago

Try it out.

bigboss97 commented 4 years ago

Try it out.

Can you give us the command? I found something in the description. But I haven't really understood.

word-dudely commented 3 years ago

Is there a better model already made for full body motion?

bigboss97 commented 3 years ago

Is there a better model already made for full body motion?

I've tried Impersonator++, not bad at all. https://youtu.be/91D1-Zl-w_8

This is the tutorial: https://youtu.be/Io1oEg7jIWA

There's a problem in that. The body is always in the centre. So it looks like the dancer is sliding, i.e. moonwalking 🤣 Here is how I fixed the sliding with green screen: https://youtu.be/DWKbb6LPiqw A lot of editing work. I wish someone can fix that.