Closed Defrost4528 closed 5 years ago
updated in latest source.
Created new section in config.ini for ugoira related configuration.
I was able to solve it by forcing ffmpeg
to wsl one.
update to fix typo and set the default config to always loop (-loop 0
).
I notice that sometimes the frame count is different compared the original ugoira in some images (for example, imageid=72165291 NSFW. Original has 150 frames, but the generated webp only have 115 frames, checked using HoneyView). I don't see any visual differences though.
with this cnfig, the zip is dowloaded, not deleted, nothing is created;
[Ugoira]
writeugoirainfo = False
createugoira = False
deletezipfile = True
creategif = False
createapng = False
deleteugoira = False
createwebm = False
createwebp = True
yep, because the webm/webp/gif/apng depends on ugoira file.
Should I set the default createUgoira
and deleteZipFile
to True and hide those option?
update to fix typo and set the default config to always loop (
-loop 0
).I notice that sometimes the frame count is different compared the original ugoira in some images (for example, imageid=72165291 NSFW. Original has 150 frames, but the generated webp only have 115 frames, checked using HoneyView). I don't see any visual differences though.
Indeed, and on the OP ugoira (68533224_ugoira1920x1080 - 金髪ボンボンお嬢ちゃん) 1 frame is added ; 121 vs 120, checked with HoneyView).
Should I set the default createUgoira and deleteZipFile to True and hide those option?
I am not sure. I think some people would want to keep the zip file, at least.
Without createUgoira
the option deleteugoira
is less clear also, for newcomer that doesn't know what ugoira is.
Oh and happy new year! ;)
the extra frame might be related to #381? https://trac.ffmpeg.org/wiki/Slideshow#Concatdemuxer
It makes sense yes.
commenting out that line apparently remove the extra frame, but it doesn't fix the missing frame issue, unless libwebp do some optimization? Setting the parameter to lossless -lossless 1 -loop 0
doesn't change the frame count.
Doing some testing, the skipped files may be because of framerate. Maybe it's somehow conflicting with the duration set in the concat demuxer.
ffmpeg -f concat -i cat.txt -vsync vfr -vf "format=yuv420p" -vcodec libwebp -lossless 0 -q:v 85 -loop 0 "out.webp"
Normal conversion with no framerate change.
Output log:
ffmpeg version N-83454-g3aae1ef Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
libavutil 55. 46.100 / 55. 46.100
libavcodec 57. 77.100 / 57. 77.100
libavformat 57. 66.101 / 57. 66.101
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 73.100 / 6. 73.100
libswscale 4. 3.101 / 4. 3.101
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Input #0, concat, from 'cat.txt':
Duration: 00:00:04.50, start: 0.000000, bitrate: 10 kb/s
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 672x1000 [SAR 15589:15608 DAR 327369:487750], 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0000000002a25b80] deprecated pixel format used, make sure you did set range correctly
Output #0, webp, to 'out.webp':
Metadata:
encoder : Lavf57.66.101
Stream #0:0: Video: webp (libwebp), yuv420p, 672x1000 [SAR 15589:15608 DAR 327369:487750], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
Metadata:
encoder : Lavc57.77.100 libwebp
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> webp (libwebp))
Press [q] to stop, [?] for help
Past duration 0.999992 too large
Last message repeated 6 times
Past duration 0.999992 too large 355kB time=00:00:00.36 bitrate=8054.9kbits/s dup=0 drop=1 speed=0.695x
Last message repeated 8 times
Past duration 0.999992 too large 704kB time=00:00:00.72 bitrate=7996.9kbits/s dup=0 drop=4 speed=0.686x
Last message repeated 8 times
Past duration 0.999992 too large 1054kB time=00:00:01.08 bitrate=7988.1kbits/s dup=0 drop=7 speed=0.68x
Last message repeated 8 times
Past duration 0.999992 too large 1404kB time=00:00:01.44 bitrate=7983.3kbits/s dup=0 drop=10 speed=0.678x
Last message repeated 8 times
Past duration 0.999992 too large 1758kB time=00:00:01.80 bitrate=7996.7kbits/s dup=0 drop=13 speed=0.676x
Last message repeated 8 times
Past duration 0.999992 too large 2115kB time=00:00:02.16 bitrate=8017.9kbits/s dup=0 drop=16 speed=0.674x
Last message repeated 8 times
Past duration 0.999992 too large 2465kB time=00:00:02.52 bitrate=8009.3kbits/s dup=0 drop=19 speed=0.675x
Last message repeated 8 times
Past duration 0.999992 too large 2821kB time=00:00:02.88 bitrate=8021.3kbits/s dup=0 drop=22 speed=0.674x
Last message repeated 8 times
Past duration 0.999992 too large 3176kB time=00:00:03.24 bitrate=8027.9kbits/s dup=0 drop=25 speed=0.674x
Last message repeated 8 times
Past duration 0.999992 too large 3528kB time=00:00:03.60 bitrate=8027.0kbits/s dup=0 drop=28 speed=0.675x
Last message repeated 8 times
Past duration 0.999992 too large 3886kB time=00:00:03.96 bitrate=8037.0kbits/s dup=0 drop=31 speed=0.674x
Last message repeated 8 times
Past duration 0.999992 too large 4236kB time=00:00:04.32 bitrate=8030.5kbits/s dup=0 drop=34 speed=0.675x
Last message repeated 4 times
frame= 114 fps= 17 q=-0.0 Lsize= 4469kB time=00:00:04.52 bitrate=8098.5kbits/s dup=0 drop=36 speed=0.668x
video:4468kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.030862%
Press Enter or Esc to close console...
RESULTING FILE HAS 114 FRAMES
ffmpeg -f concat -i cat.txt -vsync vfr -vf "format=yuv420p" -r 33.3333 -vcodec libwebp -lossless 0 -q:v 85 -loop 0 "out_r_33.3333.webp"
This time I set parameter -r to 33.3333 FPS (derived from 30ms duration)
Output log:
ffmpeg version N-83454-g3aae1ef Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
libavutil 55. 46.100 / 55. 46.100
libavcodec 57. 77.100 / 57. 77.100
libavformat 57. 66.101 / 57. 66.101
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 73.100 / 6. 73.100
libswscale 4. 3.101 / 4. 3.101
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Input #0, concat, from 'cat.txt':
Duration: 00:00:04.50, start: 0.000000, bitrate: 10 kb/s
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 672x1000 [SAR 15589:15608 DAR 327369:487750], 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0000000002ad83e0] deprecated pixel format used, make sure you did set range correctly
Output #0, webp, to 'out_r33.3333.webp':
Metadata:
encoder : Lavf57.66.101
Stream #0:0: Video: webp (libwebp), yuv420p, 672x1000 [SAR 15589:15608 DAR 327369:487750], q=2-31, 200 kb/s, 33.33 fps, 1k tbn, 33.33 tbc
Metadata:
encoder : Lavc57.77.100 libwebp
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> webp (libwebp))
Press [q] to stop, [?] for help
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 394kB time=00:00:00.33 bitrate=9753.4kbits/s speed=0.597x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large 782kB time=00:00:00.63 bitrate=10148.5kbits/s speed=0.569x
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 1129kB time=00:00:00.90 bitrate=10268.8kbits/s speed=0.556x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 1481kB time=00:00:01.17 bitrate=10360.8kbits/s speed=0.552x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 1833kB time=00:00:01.44 bitrate=10420.6kbits/s speed=0.546x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large 2185kB time=00:00:01.71 bitrate=10459.4kbits/s speed=0.545x
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 2543kB time=00:00:01.98 bitrate=10514.3kbits/s speed=0.544x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large 2932kB time=00:00:02.28 bitrate=10531.8kbits/s speed=0.543x
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666496 too large
Past duration 0.999992 too large
Past duration 0.666985 too large 3283kB time=00:00:02.55 bitrate=10542.1kbits/s speed=0.542x
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large 3639kB time=00:00:02.82 bitrate=10567.0kbits/s speed=0.541x
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large 3996kB time=00:00:03.09 bitrate=10589.8kbits/s speed=0.541x
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large 4346kB time=00:00:03.36 bitrate=10593.9kbits/s speed=0.54x
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large 4698kB time=00:00:03.63 bitrate=10599.6kbits/s speed=0.54x
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large 5056kB time=00:00:03.90 bitrate=10618.1kbits/s speed=0.54x
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large 5445kB time=00:00:04.20 bitrate=10618.7kbits/s speed=0.54x
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large
Past duration 0.999992 too large
Past duration 0.666985 too large 5795kB time=00:00:04.47 bitrate=10617.9kbits/s speed=0.54x
frame= 150 fps= 18 q=-0.0 Lsize= 5873kB time=00:00:04.50 bitrate=10689.7kbits/s speed=0.535x
video:5872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.030670%
Press Enter or Esc to close console...
OUTPUT FILE HAS 150 FRAMES
While the second test still has "Past duration 0.XXXXXX too large" errors, it seems to still output the right amount of frames.
Comparing the speeds between the APNG and WebP, they are extremely close, but the APNG seems ever so slightly faster. The gap between same events widening becomes apparent after around 30 seconds. But I'm unsure if this is result of Honeyview's interpretation of both files, a problem in the APNG, or a problem in WEBP (framerate round-off?).
Tested with this parameter -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999
based on https://superuser.com/a/1073260, looks good.
EDIT: tested for both webm and webp codec. Updated in latest commit https://github.com/Nandaka/PixivUtil2/commit/aedf996e68108906738d752d35e864e3737326fc
btw, do you have any suggestion on converting olds ugoira? Reading your commits i found https://github.com/tsudoko/ugoira-tools/blob/master/ugoira2webm/ugoira2webm.py
Not sure if adding an option in your app is necessary: it's a oneshot process and not in the same scope. What do you think?
@whinette normally I just rerun the download process and it will try to reprocess the ugoira file without downloading, assuming the config.ini is configured correctly (combination of alwayscheckfilesize = True
, createugoira = True
, and creategif/apng/webm/webp = True
).
Else, I can provide one-off script (not executables) to convert the ugoira to webm/webp.
Some of the ugira doesn't exist anymore, I'll go the script route :)
Try this script, you can change the root_dir
parameter
Trying it now on a sample folder, it looks well. \
Had to replace line 169 with fullpath = u"{0}\\{1}".format(toUnicode(root), toUnicode(f))
@ running on linux, with os.path.join()
I made many test and I think I found a slight problem. I think the last frame isn't played as long as it should. this ugoira is the best/worst example : \
webp and ugoira: https://www.dropbox.com/sh/3zsojkcr4zsx97x/AACC7Boqteb6G1XMZCywGNdNa?dl=0 \
webp generated with "-threads 3 -vf 'format=yuv420p' -lossless 0 -q:v 85 -loop 0 -vsync 2 -r 999"
The last frame still played, but looks like it ignore the timestamp. Most likely the same issue with https://github.com/Nandaka/PixivUtil2/issues/381.
I can remove the if condition at https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L827, but this will increase the frame count for webp, compared to the original ugoira.
I think it is fair.
Also, can you change the st_mtime of the generated file to the pixiv original file, like it is done for other standard downloaded files?
Thanks again. :)
so, should I add the extra frame?
As for the timestamp, why you want to do this for the generated webm/webp? I understand if it was for the generated ugoira.
Yes, to have the same timing as the original animation.
In my case, the webm/webp will simply replace the ugoira, the pixiv submission date is important to me for sorting and other stuff (stats etc)
It works perfectly, thanks!
After some thought and doing some limited testing, I'd like to suggest WebP as an alternative to APNG and GIF. It has efficiency similar to WebM, while being easier to loop (essential in many ugoira) and enjoying the flexibility of images.
_I'll be using this Pixiv ugoira as a sample._
Compatibility
Performance
Lossy mode Q100: 31.8 MB indiscernible from lossless, 4x smaller than APNG Q 90: 16.7 MB almost no visible artifacts, 7x smaller Q 80: 9.84 MB some visible artifacts, 12x smaller Q 50: 5.96 MB apparent artifacts but bearable, 20x smaller Q 10: 2.80 MB extreme artifacts and not recommended, 42x smaller
Command syntax used:
ffmpeg -i %1 -vf format=yuv420p -vcodec libwebp -lossless 0 -q:v [quality factor here] -loop 0 "%~n1_lossy.webp"
Based on results, this is my conclusion:
For implementation, I think the best way to do this is to have a "createwebp" option and include a second pair of ffmpeg options in the config just like WebM, specifically for WebP. As for the default parameters, I'd go with a factor of 85. But of course, it depends on you. Here are the lossy files so you can compare and decide for yourself.
Thank you for taking your time to read this!