Nandaka / PixivUtil2

Download images from Pixiv and more!
http://nandaka.devnull.zone/
BSD 2-Clause "Simplified" License
2.31k stars 250 forks source link

Questions about Ugoiras: What are the best parameters to convert them? #1214

Open DisasterInbound opened 1 year ago

DisasterInbound commented 1 year ago

Hello. I noticed the ffmpeg parameters are different from the default ones shown on the Readme file. I remember updating them a few times in the past when it was mentioned or recommended in a couple of issues here or also in the release notes however I'm uncertain If they are the "best" parameters.

My config.ini file is set up like this:

[FFmpeg] ffmpeg = ffmpeg.exe ffmpegCodec = libvpx-vp9 ffmpegExt = webm ffmpegParam = -lossless 1 -vsync 2 -r 999 -pix_fmt yuv420p webpCodec = libwebp webpParam = -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999 gifParam = -filter_complex [0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle apngParam = -vf setpts=PTS-STARTPTS,hqdn3d=1.5:1.5:6:6 -plays 0 verboseOutput = False

[Ugoira] writeUgoiraInfo = False createUgoira = True deleteZipFile = True createGif = False createApng = False deleteUgoira = True createWebm = True createWebp = False

I don't have knowledge on how to set them up properly so I would like to ask your help with these questions I have:

  1. Are the default parameters mentioned on Readme the best parameters (no loss of quality/bitrate) to convert ugoiras?

  2. I only use createWebm because from my understanding: gif files would be bloated and possibly have less image/color quality than the original file, apng would also create bloated files and for some reason some apng are not playable on my PC (either they are too heavy or maybe something specific to my own PC that I never investigated). I never used Webp files before so I have a neutral opinion on them. If webm files are not recommended for ugoiras, which option would be best without (losing too much or none) of image quality/bitrate?

  3. Most likely this is related to my current parameters but sometimes I'm unable to play some of those ugoiras on mpv.net. I reported the issue to them and one person mentioned the files have 999 fps which seems to break the video player. What parameter or setting is the best to prevent playback issues or to add a fps limiter or similar so that they can be played properly?

  4. If I need to update the parameters on config.ini, should I also then use the option u. Ugoira re-encode to re-download and re-convert all the webm files I have? Or is this option used for a very different and specific reason?

Any other advice would be helpful. Thanks in advance.

PatrickL546 commented 1 year ago
  1. Use the current default settings as it's the 'best'

    [FFmpeg]
    ffmpeg = ffmpeg.exe
    ffmpegCodec = libvpx-vp9
    ffmpegExt = webm
    ffmpegParam = -lossless 0 -crf 15 -b 0 -vsync 0
    webpCodec = libwebp
    webpParam = -lossless 0 -compression_level 5 -quality 100 -loop 0 -vsync 0
    gifParam = -filter_complex [0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle -vsync 0
    apngParam = -plays 0 -vsync 0
    verboseOutput = False
  2. Webm is free and open source and it's very efficient, Webp is the image version of it. Use Webm for ugoira

  3. I use K-lite codec pack with MPC-HC and had no trouble playing it. I'd say re-encoding it with new settings would fix it.

  4. Re-encoding it would use the .ugoira/(not sure if it does zip files) already downloaded and encode it with the settings in config.ini

L4cache commented 1 year ago

the best way is to not convert them, just use something like honeyview that can read .ugoira animation, or pack it in mkv like gallery-dl does.

DisasterInbound commented 1 year ago

1,2 and 3

Thanks I'll try them out.

  1. Re-encoding it would use the .ugoira/(not sure if it does zip files) already downloaded and encode it with the settings in config.ini

I have around 5k ugoira files (between webm and gif files I downloaded in the past) and I don't mind downloading them again. Do you think this options will do both downloading and converting them if the .ugoira is no longer on PC?

DisasterInbound commented 1 year ago

the best way is to not convert them, just use something like honeyview that can read .ugoira animation, or pack it in mkv like gallery-dl does.

Thanks. I'm also using HoneyView but I'm trying to discontinue using it since it no longer has active support besides security fixes. So it won't be of much use to me if I kept it only for this type of files.

PatrickL546 commented 1 year ago

This is gallery-dl arguments ffmpeg args on .ugoira. Might try them out later and see the quality and space usage

    postprocessor.add_argument(
        "--ugoira-conv",
        dest="postprocessors", action="append_const", const={
            "name"          : "ugoira",
            "ffmpeg-args"   : ("-c:v", "libvpx", "-crf", "4", "-b:v", "5000k"),
            "ffmpeg-twopass": True,
            "whitelist"     : ("pixiv", "danbooru"),
        },
        help="Convert Pixiv Ugoira to WebM (requires FFmpeg)",
    )
    postprocessor.add_argument(
        "--ugoira-conv-lossless",
        dest="postprocessors", action="append_const", const={
            "name"          : "ugoira",
            "ffmpeg-args"   : ("-c:v", "libvpx-vp9", "-lossless", "1",
                               "-pix_fmt", "yuv420p"),
            "ffmpeg-twopass": False,
            "whitelist"     : ("pixiv", "danbooru"),
        },
        help="Convert Pixiv Ugoira to WebM in VP9 lossless mode",
    )
    postprocessor.add_argument(
        "--ugoira-conv-copy",
        dest="postprocessors", action="append_const", const={
            "name"             : "ugoira",
            "extension"        : "mkv",
            "ffmpeg-args"      : ("-c:v", "copy"),
            "ffmpeg-twopass"   : False,
            "repeat-last-frame": False,
            "whitelist"        : ("pixiv", "danbooru"),
        },
        help="Convert Pixiv Ugoira to MKV without re-encoding any frames",
    )
ManueLink64 commented 1 year ago

Hi, this is my personal configuration (only webm, no ugoira, no zip), saves a ton of space and the quality is near to lossless, I know the upscaled color space is a bit overkill, but reduces the color banding. You can still use yuv420p or yuv422p

[FFmpeg]
ffmpeg = ffmpeg.exe
ffmpegCodec = libvpx-vp9
ffmpegExt = webm
ffmpegParam = -row-mt 1 -deadline good -crf 19 -vsync 2 -r 999 -pix_fmt yuv444p
webpCodec = libwebp
webpParam = -row-mt 1 -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999
gifParam = -filter_complex "[0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle"
apngParam = -vf "setpts=PTS-STARTPTS,hqdn3d=1.5:1.5:6:6" -plays 0
verboseOutput = True

[Ugoira]
writeUgoiraInfo = False
createUgoira = True
deleteZipFile = True
createGif = False
createApng = False
deleteUgoira = True
createWebm = True
createWebp = False
PatrickL546 commented 1 year ago

I was able to run the first 2 gallery dl options and I got this. Wasn't able to run Convert Pixiv Ugoira to MKV without re-encoding any frames can't figure out if i could run it on local files

Top 2mb is gallery-dl Convert Pixiv Ugoira to WebM. It looks blocky Middle 49mb gallery-dl Convert Pixiv Ugoira to WebM in VP9 lossless mode and is twice the size of ugoira and looks good Bottom 6mb is Pixivutil2, honestly looks a lot like the lossless one. And it looks quite good without much file size.

image

L4cache commented 1 year ago

I was able to run the first 2 gallery dl options and I got this. Wasn't able to run Convert Pixiv Ugoira to MKV without re-encoding any frames can't figure out if i could run it on local files

Top 2mb is gallery-dl Convert Pixiv Ugoira to WebM. It looks blocky Middle 49mb gallery-dl Convert Pixiv Ugoira to WebM in VP9 lossless mode and is twice the size of ugoira and looks good Bottom 6mb is Pixivutil2, honestly looks a lot like the lossless one. And it looks quite good without much file size.

image

when i was talking about gallery-dl I only said that the MKV method is good, not others. however, i haven't tried that feature in gallery-dl so i don't know if it was working or not. but i did same thing before (manually) and the concept is totally good. maybe i'll write a script to search .ugoira files and convert them because i think i know video/image codecs and containers better.

L4cache commented 1 year ago

https://github.com/L4cache/ugoira-into-mkv This would do

PatrickL546 commented 1 year ago

Here it is https://github.com/Nandaka/PixivUtil2/pull/1218 Works with new download and Re-encoding

DisasterInbound commented 1 year ago

Thanks everyone for their comments.

Here it is #1218 Works with new download and Re-encoding

With this the webm files will be re-downloaded and converted to MKV, right? (using the Re-encode Ugoira option) Since this PR is now merged I'll wait until it is added to the next release and try it out.

skulkexpert commented 1 year ago

Hi, this is my personal configuration (only webm, no ugoira, no zip), saves a ton of space and the quality is near to lossless, I know the upscaled color space is a bit overkill, but reduces the color banding. You can still use yuv420p or yuv422p

[FFmpeg]
ffmpeg = ffmpeg.exe
ffmpegCodec = libvpx-vp9
ffmpegExt = webm
ffmpegParam = -row-mt 1 -deadline good -crf 19 -vsync 2 -r 999 -pix_fmt yuv444p
webpCodec = libwebp
webpParam = -row-mt 1 -lossless 0 -q:v 90 -loop 0 -vsync 2 -r 999
gifParam = -filter_complex "[0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle"
apngParam = -vf "setpts=PTS-STARTPTS,hqdn3d=1.5:1.5:6:6" -plays 0
verboseOutput = True

[Ugoira]
writeUgoiraInfo = False
createUgoira = True
deleteZipFile = True
createGif = False
createApng = False
deleteUgoira = True
createWebm = True
createWebp = False

Since you seem to use webm, do you know of any convenient image viewer that will display both webm and images like jpg and png? Most image viewers support gif, but gifs are much worse than webm ...

PatrickL546 commented 1 year ago

@skulkexpert Ifranview. It also supports plugins to extend usability

Nandaka commented 1 year ago

honeyview (https://en.bandisoft.com/honeyview/) support ugoira, so no need to convert the files (unless you want to have smaller size as video codec has better compression). I use that one.

DisasterInbound commented 1 year ago

honeyview (https://en.bandisoft.com/honeyview/) support ugoira, so no need to convert the files (unless you want to have smaller size as video codec has better compression). I use that one.

Thanks, but since Honeyview is no longer being developed (other than for security fixes every few months) I'm concerned it will stop working for any reason.

DisasterInbound commented 1 year ago

@PatrickL546 thanks for the help. I've been trying the new mkv format and the files are working pretty well so far. I'm still going to try a few more times and then I'll proceed to re-convert all previous webm files.

28598519a commented 1 year ago
  1. Use the current default settings as it's the 'best'. I'm not an encoder, and that stuff is pretty complicated for what is 'best'.
[FFmpeg]
ffmpeg = ffmpeg.exe
ffmpegCodec = libvpx-vp9
ffmpegExt = webm
ffmpegParam = -lossless 0 -crf 15 -b 0 -vsync 0
webpCodec = libwebp
webpParam = -lossless 0 -compression_level 5 -quality 100 -loop 0 -vsync 0
gifParam = -filter_complex [0:v]split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle -vsync 0
apngParam = -plays 0 -vsync 0
verboseOutput = False
  1. Webm is free and open source and it's very efficient, Webp is the image version of it. Use Webm for ugoira
  2. I use K-lite codec pack with MPC-HC and had no trouble playing it. I'd say re-encoding it with new settings would fix it.
  3. Re-encoding it would use the .ugoira/(not sure if it does zip files) already downloaded and encode it with the settings in config.ini

angParam = -plays 0 -vsync 0 Sometimes it will get Non-monotonous DTS in output stream error. (example: pixiv image id: 70683660)

apngParam = -vf setpts=PTS-STARTPTS,hqdn3d=1.5:1.5:6:6 -plays 0 This works well.