search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.53k stars 3.22k forks source link

error accur when i run demo.py on mayun.mp4 and yyp.jpeg #22

Open CarolinGao opened 4 years ago

CarolinGao commented 4 years ago

This is my data. mayun.mp4.zip yyp

I run python crop-video.py --inp mayun.mp4.

Then I run python demo.py --config config/vox-256.yaml --driving_video ./mayun.mp4 --source_image ./yyp.jpeg --checkpoint ./vox-cpk.pth.tar --relative --adapt_scale.

It appers that Traceback (most recent call last): File "demo.py", line 120, in <module> driving_video = imageio.mimread(opt.driving_video, memtest=False) File "/home/user/anaconda3/envs/first-order-model/lib/python3.7/site-packages/imageio/core/functions.py", line 286, in mimread for im in reader: File "/home/user/anaconda3/envs/first-order-model/lib/python3.7/site-packages/imageio/core/format.py", line 397, in iter_data im, meta = self._get_data(i) File "/home/user/anaconda3/envs/first-order-model/lib/python3.7/site-packages/imageio/plugins/ffmpeg.py", line 396, in _get_data result, is_new = self._read_frame() File "/home/user/anaconda3/envs/first-order-model/lib/python3.7/site-packages/imageio/plugins/ffmpeg.py", line 585, in _read_frame s, is_new = self._read_frame_data() File "/home/user/anaconda3/envs/first-order-model/lib/python3.7/site-packages/imageio/plugins/ffmpeg.py", line 571, in _read_frame_data raise CannotReadFrameError(fmt % (self._pos, err1, err2)) imageio.core.format.CannotReadFrameError: Could not read frame 1750: Frame is 0 bytes, but expected 6220800. === stderr === ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 7.3.0 (crosstool-NG 1.23.0.449-a04d0) configuration: --prefix=/home/user/anaconda3/envs/first-order-model --cc=/home/conda/feedstock_root/build_artifacts/ffmpeg_1566210161358/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc --disable-doc --disable-openssl --enable-avresample --enable-gnutls --enable-gpl --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-libx264 --enable-pic --enable-pthreads --enable-shared --enable-static --enable-version3 --enable-zlib --enable-libmp3lame libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/user/projects/first-order-model/mayun.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom creation_time : 2020-03-18T14:18:57.000000Z Duration: 00:01:09.48, start: 0.000000, bitrate: 2095 kb/s Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 1962 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default) Metadata: creation_time : 2020-03-18T14:30:35.000000Z handler_name : VideoHandler encoder : AVC Coding Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 131 kb/s (default) Metadata: creation_time : 2020-03-18T14:30:35.000000Z handler_name : SoundHandler Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native)) Press [q] to stop, [?] for help Output #0, image2pipe, to 'pipe:': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom encoder : Lavf58.29.100 Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 1492992 kb/s, 30 fps, 30 tbn, 30 tbc (default) ... showing only last few lines ... major_brand : mp42 minor_version : 0 compatible_brands: mp41isom encoder : Lavf58.29.100 Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 1492992 kb/s, 30 fps, 30 tbn, 30 tbc (default) Metadata: creation_time : 2020-03-18T14:30:35.000000Z handler_name : VideoHandler encoder : Lavc58.54.100 rawvideo frame= 58 fps=0.0 q=-0.0 size= 352350kB time=00:00:01.93 bitrate=1492992.3kbits/s speed=3.86x frame= 128 fps=128 q=-0.0 size= 777600kB time=00:00:04.26 bitrate=1492991.9kbits/s speed=4.26x frame= 199 fps=132 q=-0.0 size= 1208925kB time=00:00:06.63 bitrate=1492992.1kbits/s speed=4.41x frame= 271 fps=135 q=-0.0 size= 1646325kB time=00:00:09.03 bitrate=1492992.1kbits/s speed= 4.5x frame= 342 fps=136 q=-0.0 size= 2077650kB time=00:00:11.40 bitrate=1492992.0kbits/s speed=4.55x frame= 414 fps=138 q=-0.0 size= 2515050kB time=00:00:13.80 bitrate=1492992.0kbits/s speed=4.59x frame= 486 fps=139 q=-0.0 size= 2952450kB time=00:00:16.20 bitrate=1492992.0kbits/s speed=4.62x frame= 555 fps=138 q=-0.0 size= 3371625kB time=00:00:18.50 bitrate=1492992.0kbits/s speed=4.61x frame= 625 fps=138 q=-0.0 size= 3796875kB time=00:00:20.83 bitrate=1492992.0kbits/s speed=4.62x frame= 694 fps=138 q=-0.0 size= 4216050kB time=00:00:23.13 bitrate=1492992.0kbits/s speed=4.61x frame= 764 fps=138 q=-0.0 size= 4641300kB time=00:00:25.46 bitrate=1492992.0kbits/s speed=4.61x frame= 834 fps=138 q=-0.0 size= 5066550kB time=00:00:27.80 bitrate=1492992.0kbits/s speed=4.62x frame= 904 fps=139 q=-0.0 size= 5491800kB time=00:00:30.13 bitrate=1492992.0kbits/s speed=4.62x frame= 975 fps=139 q=-0.0 size= 5923125kB time=00:00:32.50 bitrate=1492992.0kbits/s speed=4.62x frame= 1045 fps=139 q=-0.0 size= 6348375kB time=00:00:34.83 bitrate=1492992.0kbits/s speed=4.62x frame= 1116 fps=139 q=-0.0 size= 6779700kB time=00:00:37.20 bitrate=1492992.0kbits/s speed=4.62x frame= 1180 fps=138 q=-0.0 size= 7168500kB time=00:00:39.33 bitrate=1492992.0kbits/s speed= 4.6x frame= 1250 fps=138 q=-0.0 size= 7593750kB time=00:00:41.66 bitrate=1492992.0kbits/s speed= 4.6x frame= 1316 fps=138 q=-0.0 size= 7994700kB time=00:00:43.86 bitrate=1492992.0kbits/s speed=4.59x frame= 1388 fps=138 q=-0.0 size= 8432100kB time=00:00:46.26 bitrate=1492992.0kbits/s speed= 4.6x frame= 1460 fps=138 q=-0.0 size= 8869500kB time=00:00:48.66 bitrate=1492992.0kbits/s speed=4.61x frame= 1533 fps=139 q=-0.0 size= 9312975kB time=00:00:51.10 bitrate=1492992.0kbits/s speed=4.62x frame= 1597 fps=138 q=-0.0 size= 9701775kB time=00:00:53.23 bitrate=1492992.0kbits/s speed= 4.6x frame= 1667 fps=138 q=-0.0 size=10127025kB time=00:00:55.56 bitrate=1492992.0kbits/s speed= 4.6x frame= 1725 fps=137 q=-0.0 size=10479375kB time=00:00:57.50 bitrate=1492992.0kbits/s speed=4.57x

CarolinGao commented 4 years ago

I am confused whether the video is too large. I sugget u to open the video I upload because I dont know if the vox-cpk.pth.tar can be used in such video. And If the video has more than one person, which one will it replace with the given face？

AliaksandrSiarohin commented 4 years ago

Both image and video should be cropped. Crop-video.py produce just a crop suggestions. So you can produce actual crop using ffmpeg command, that was printed by Crop-video.py.

It may be that it does not produce anything, in this case there is a lot of movement in video and it is hard to produce a single crop. Try to decrease min_frames.

CodingMice commented 4 years ago

Both image and video should be cropped. Crop-video.py produce just a crop suggestions. So you can produce actual crop using ffmpeg command, that was printed by Crop-video.py.

It may be that it does not produce anything, in this case there is a lot of movement in video and it is hard to produce a single crop. Try to decrease min_frames.

It seems that it is easy to get None result from Crop-video.py if the face dose not in the center of the frame.

AliaksandrSiarohin commented 4 years ago

I don't think it related to the position of the face. Can you provide an example?