AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.35k stars 3.19k forks source link

Any chance we could get a Prebuilt Windows Release with dependencies included? #35

Open ZeroCool940711 opened 4 years ago

ZeroCool940711 commented 4 years ago

It would be nice if we could get a release with everything included to run the code or at least run the demo, something similar to this , there is a folder called _internal with a python virtualenv and all the dependencies to run that DeepFake project, even scripts to do specific tasks fast like training. Currently its hard to run the demo for First Order Motion Model for Image Animation as it has some dependencies that need a specific version of some libraries, in my case I tried to run the demo and it didnt work, after a few hours downloading all the necessary stuff and datasets I couldnt run it no matter what I tried, would be nice to have something we can test at least the demo right away and then we can decide what to continue testing.

AliaksandrSiarohin commented 4 years ago

Have you tried to use google-colab notebook demo.ipynb? You can run a demo on google servers, and nothing need to be installed.

ZeroCool940711 commented 4 years ago

I dont really like Colab and I actually have a really potent computer so I would rather run it on my own computer, not only it will be faster it will also be safer to run it on my own computer.

AliaksandrSiarohin commented 4 years ago

I can hardly help you, since I don't have any windows machine available.

WilliamOConnell commented 4 years ago

I was able to get it running on Windows. Make sure you're using 64-bit Python, then install the PyTorch 1.0.0 wheel from here. Then you can use pip for the rest of requirements.txt.

ZeroCool940711 commented 4 years ago

@WilliamOConnell thanks for the info, at least now I know someone was able to make it run on windows so I can try again later, also can you share the exact steps you used on windows to make it run?, if its not a problem of course I would appreciate any extra information.

WilliamOConnell commented 4 years ago

I downloaded the checkpoint for VoxCeleb (vox-cpk.pth.tar) and put it in a folder called checkpoints/. Then I made a virtualenv, installed the wheel for PyTorch, and did pip install -r requirements.txt to install everything else. I had some bumps along the way (needed to install 64-bit Python as I mentioned) but that was about it.

I was a little confused by the instructions about how to run it so I just copied off the Colab to make my own .py file that looks something like this:

import imageio
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from skimage.transform import resize
import warnings
warnings.filterwarnings("ignore")

source_image = imageio.imread('mycontent/source.png')
driving_video = imageio.mimread('mycontent/driver.mp4')

source_image = resize(source_image, (256, 256))[..., :3]
driving_video = [resize(frame, (256, 256))[..., :3] for frame in driving_video]

from demo import load_checkpoints
generator, kp_detector = load_checkpoints(config_path='config/vox-256.yaml', checkpoint_path='checkpoints/vox-cpk.pth.tar')

from demo import make_animation
from skimage import img_as_ubyte

predictions = make_animation(source_image, driving_video, generator, kp_detector, relative=True)

#save resulting video
imageio.mimsave('generated.mp4', [img_as_ubyte(frame) for frame in predictions])

Obviously you should change the paths for the video files and such to wherever you put them.

ZeroCool940711 commented 4 years ago

I see, thanks @WilliamOConnell , im downloading again all the datasets and all the necessary files so I have everything ready, I will try everything in an hour or two, will let you know if it works and if it does I will try to create a virtualenv with some scripts for windows so its "plug and play" and comes with everything included where the only thing you have to install is your video card driver just like the DeepFakeLab link I put on my first comment.

ZeroCool940711 commented 4 years ago

Hey guys, just an update, today when I woke up I tried again following the steps mentioned before by WilliamOConnell and I managed to run the demo on Windows 10, I did get a lot of warnings, some of them were because my two GPUs were different but thats not a problem for what I saw, also had to update numpy to the latest version as the one on the requirements.txt gave me some errors but after updating it did work, I recommend you guys change the requirements.txt file so pip installs the most recent version of numpy instead. Im going to start learning how everything works on the code and how to use it correctly and after that I'll see if I can help with some improvements.

ZeroCool940711 commented 4 years ago

@AliaksandrSiarohin Im opening a new issue (#43) to collect all the bugs that are specific to Windows, the goal is to gather there all the windows specific issues or have some comments pointing to issues that happen on windows but not on Linux so its easier for you or someone else to fix those bugs, if you could pin that issue so it appears on the top of the issue tab when someone go to check the issue list it will be easier for people to check there for known problems or solutions on Window.

AncientSion commented 4 years ago

Please keep us posted, i would love to try this out on my w10.

ZeroCool940711 commented 4 years ago

@AncientSion are you getting some errors when running it on windows? if you are getting some errors post it on Issue #43 or here so I can try to fix it, im currently fixing a log folder issue where the name has characters that are not allowed on windows, I've already fixed a PyYAML yaml.load(input) deprecation warning that happens on windows and not sure if also happens on linux and created a PR for it but have to wait until the repository owner accept it.

ZeroCool940711 commented 4 years ago

@WilliamOConnell have you tried training on Windows? you said the demo ran for you without any problem but have you tried training? if you did, did it run without showing any error or it didnt work for you? im asking because after I managed to run the demo I cant seem to be able to train with my own dataset, the run.py script always show some errors, the last one I was getting was about ffmpeg not reading some frames, Im trying to fix it but no luck so far.

python run.py --config config/test.yaml --device_ids 0,1
Use predefined train-test split.
Training...
D:\Python\lib\site-packages\torch\nn\parallel\data_parallel.py:25: UserWarning:
    There is an imbalance between your GPUs. You may want to exclude GPU 1 which
    has less than 75% of the memory or cores of GPU 0. You can do so by setting
    the device_ids argument to DataParallel, or by setting the CUDA_VISIBLE_DEVICES
    environment variable.
  warnings.warn(imbalance_warn.format(device_ids[min_pos], device_ids[max_pos]))
  0%|                                                                                                      | 0/100 [00:00<?, ?it/s]D:\Python\lib\site-packages\imageio\core\format.py:403: UserWarning: Could not read last frame of first-order-model-master\data\test\train\91pn9FUZ9NS.mp4.
  warn('Could not read last frame of %s.' % uri)
Traceback (most recent call last):
  File "run.py", line 77, in <module>
    train(config, generator, discriminator, kp_detector, opt.checkpoint, log_dir, dataset, opt.device_ids)
  File "D:\Alejandro\Projects\Python\Otros\Machine Learning\DeepFakes\first-order-model-master\train.py", line 50, in train
    for x in dataloader:
  File "D:\Python\lib\site-packages\torch\utils\data\dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "D:\Python\lib\site-packages\torch\utils\data\dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
imageio.core.format.CannotReadFrameError: Traceback (most recent call last):
  File "D:\Python\lib\site-packages\imageio\plugins\ffmpeg.py", line 565, in _read_frame_data
    (len(s), framesize))
RuntimeError: Frame is 0 bytes, but expected 2030400.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Python\lib\site-packages\torch\utils\data\dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "D:\Python\lib\site-packages\torch\utils\data\dataloader.py", line 138, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
 first-order-model-master\frames_dataset.py", line 43, in read_video
    video = np.array(mimread(name, memtest=False))
  File "D:\Python\lib\site-packages\imageio\core\functions.py", line 286, in mimread
    for im in reader:
  File "D:\Python\lib\site-packages\imageio\core\format.py", line 397, in iter_data
    im, meta = self._get_data(i)
  File "D:\Python\lib\site-packages\imageio\plugins\ffmpeg.py", line 396, in _get_data
    result, is_new = self._read_frame()
  File "D:\Python\lib\site-packages\imageio\plugins\ffmpeg.py", line 585, in _read_frame
    s, is_new = self._read_frame_data()
  File "D:\Python\lib\site-packages\imageio\plugins\ffmpeg.py", line 571, in _read_frame_data
    raise CannotReadFrameError(fmt % (self._pos, err1, err2))
imageio.core.format.CannotReadFrameError: Could not read frame 347:
Frame is 0 bytes, but expected 2030400.
WilliamOConnell commented 4 years ago

No, I didn't try training it. Can't help there, sorry.

dewijones92 commented 4 years ago

try docker https://github.com/AliaksandrSiarohin/first-order-model/pull/55

Artchom commented 4 years ago

Have you tried to use google-colab notebook demo.ipynb? You can run a demo on google servers, and nothing need to be installed.

yes lots of errors fighting to make it work

cagneymoreau commented 4 years ago

I can confirm I got it working on windows 10 after a bit of struggle. 64 bit python was the main hangup. I also had to re install several files that were presented in obvious error message requests from python even after the requirments file. I used the pytorch website to download the new pytorch https://pytorch.org/

Another odd thing is after running the script it took a full minute before any feed back happened so be patient