Sorry everyone, but I'm having a pretty fundamental problem and I'm stuck

[mactac]

I'm trying to run the code here: https://colab.research.google.com/github/AliaksandrSiarohin/first-order-model/blob/master/demo.ipynb

It works great, but I simply cannot figure out how to replace the images and video files with my own.

It uses a shared file folder on my Google drive, and I cannot add files or make changes to a shared folder of someone else's.

If I copy the whole folder and re-upload it to my google drive, it cannot find it anymore as it's expecting to look for whatever folder was set up in the second step ("mount your google drove folder"... I tried naming the folder exactly the same as the shared folder, but it cannot find that folder anymore.

Can anyone give me a basic step-by-step on how I can execute the demo from the website I linked with my own source images/videos without having to rewrite the code?, I'd really appreciate it.

I seem to be missing something very fundamental here, as it seems to be set up to use your own files, but it doesn't seem to have the capability of actually doing it without modifications to the code.

thanks !

[AliaksandrSiarohin]

On the right in colab you can find the window which shows you, your local file system. You can upload your images and videos directly to your colab file system. You will need to change the path to source image and driving video accordingly.

[mactac]

On the right in colab you can find the window which shows you, your local file system

Do mean on this page: https://colab.research.google.com/github/AliaksandrSiarohin/first-order-model/blob/master/demo.ipynb

I cannot see a window. Apologies, I've never use Colab before.

[AliaksandrSiarohin]

Sorry left.

[mactac]

Actually, I think I might have figured it out - changed the path in the code in the last block.. But now I'm getting:

RuntimeError Traceback (most recent call last) /usr/local/lib/python3.6/dist-packages/imageio/plugins/ffmpeg.py in _read_frame_data(self) 620 raise RuntimeError( --> 621 "Frame is %i bytes, but expected %i." % (len(s), framesize) 622 )

RuntimeError: Frame is 0 bytes, but expected 196608.

During handling of the above exception, another exception occurred:

CannotReadFrameError Traceback (most recent call last) 5 frames /usr/local/lib/python3.6/dist-packages/imageio/plugins/ffmpeg.py in _read_frame_data(self) 626 err2 = self._stderr_catcher.get_text(0.4) 627 fmt = "Could not read frame %i:\n%s\n=== stderr ===\n%s" --> 628 raise CannotReadFrameError(fmt % (self._pos, err1, err2)) 629 return s, is_new 630

CannotReadFrameError: Could not read frame 586: Frame is 0 bytes, but expected 196608.

I assume the video is the wrong format? Anything I need to do specifically?

Also, some questions:

1. I see some references to audio, but I don't hear any audio in the examples. Anything I need to do to make audio work?
2. It seems to be limited to a certain image and video size. Is this somethign that can be changed, or is it fixed?

[mactac]

Any help with this error would be super appreciated, I'm trying to do something with this for my friend's birthday tomorrow :) I've tried it with a ton of different encoding schemes and videos, always the same result.

[AliaksandrSiarohin]

Try to convert using

!ffmpeg -i /content/gdrive/My\ Drive/first-order-motion-model/07.mkv hinton.mp4

[mactac]

Try to convert using

!ffmpeg -i /content/gdrive/My\ Drive/first-order-motion-model/07.mkv hinton.mp4

Thanks. Since I don't have Linux installed, I assume that I would need to find a machine to do it on... or can I add something to the code somewhere for this?

[AliaksandrSiarohin]

Use colab.

[mactac]

Use colab.

OK, I will try to figure out how to do that. Thanks!

[mactac]

Thanks - got it working. I see references to audio in the code, is audio preserved from the source video? I don't see it in the result video that is shown.

Also, does it support higher resolution at all?

Thanks!

[AliaksandrSiarohin]

Search for related issues.

[tgohblio]

Thanks - got it working. I see references to audio in the code, is audio preserved from the source video? I don't see it in the result video that is shown.

Also, does it support higher resolution at all?

Thanks!

Hey mactac,

You can extract audio from original video and apply it to your generated output easily using moviepy library. See my reference below: https://colab.research.google.com/drive/1T2BEp281ogKwrH5MbRWH1IbU74XehNL_#scrollTo=Qnaa6d2PQ9wp

Do note that you need to get the "fps" first, so your audio is in sync with the video: https://colab.research.google.com/drive/1T2BEp281ogKwrH5MbRWH1IbU74XehNL_#scrollTo=fdFdasHEj3t7

[tandav]

same problem. Using scikit-video to load video instead of imageio fixes the problem for me

!pip install scikit-video
import skvideo.io  
driving_video = skvideo.io.vread("/content/gdrive/My Drive/first-order-motion-model/input_video.mp4") 
print(driving_video.shape)