search

master-of-zen / Av1an

Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding
GNU General Public License v3.0
1.4k stars 147 forks source link

Different target quality Q 0.3.1 vs 0.4.0 #711

Closed doits closed 1 year ago

doits commented 1 year ago

After upgrading to 0.4.0 I noticed the same target quality VMAF gives much lower Q. Input is a 4K 50fps HDR video of a camcorder.

This is my command line:

av1an -i input.mp4 -v "--cpu-used=2 --end-usage=q --cq-level=18 --threads=8 --color-primaries=bt2020 --transfer-characteristics=smpte2084 --matrix-coefficients=bt2020ncl --lag-in-frames=48 --bit-depth=10 --kf-max-dist=500 --disable-kf --sharpness=1 --tile-columns=1" -a "-dn -c:a libvorbis -q:a 7" -x 500 -w 24 --sc-downscale-height 720 --target-quality 97 --log-level debug -o output.webm

0.3.1

INFO [av1an_core::settings] temporary directory: .926b233
INFO [av1an_core::settings] scenecut: found 14 scene(s) [with extra_splits(500 frames): 14]
INFO [av1an_core::settings] encoding resumed with 0/14 chunks completed (14 remaining)
DEBUG [av1an_core::broker]  started chunk 00001: 473 frames
DEBUG [av1an_core::broker]  started chunk 00011: 473 frames
DEBUG [av1an_core::broker]  started chunk 00012: 473 frames
DEBUG [av1an_core::broker]  started chunk 00004: 473 frames
DEBUG [av1an_core::broker]  started chunk 00005: 473 frames
DEBUG [av1an_core::broker]  started chunk 00006: 473 frames
DEBUG [av1an_core::broker]  started chunk 00008: 473 frames
DEBUG [av1an_core::broker]  started chunk 00009: 473 frames
DEBUG [av1an_core::broker]  started chunk 00002: 473 frames
DEBUG [av1an_core::broker]  started chunk 00013: 473 frames
DEBUG [av1an_core::broker]  started chunk 00000: 472 frames
DEBUG [av1an_core::broker]  started chunk 00003: 472 frames
DEBUG [av1an_core::broker]  started chunk 00007: 472 frames
DEBUG [av1an_core::broker]  started chunk 00010: 472 frames
INFO [av1an_core::target_quality] chunk 00000: P-Rate=4, 472 frames
INFO [av1an_core::target_quality] chunk 00000: TQ-Probes: [(97.12, 15), (97.03, 17), (95.73, 35)]
INFO [av1an_core::target_quality] chunk 00000: Target Q=17, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00004: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00004: TQ-Probes: [(97.22, 15), (97.02, 19), (95.91, 35)]
INFO [av1an_core::target_quality] chunk 00004: Target Q=19, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00001: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00001: TQ-Probes: [(97.22, 15), (96.97, 18), (95.18, 35)]
INFO [av1an_core::target_quality] chunk 00001: Target Q=17, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00013: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00013: TQ-Probes: [(97.23, 15), (97.05, 17), (94.11, 35)]
INFO [av1an_core::target_quality] chunk 00013: Target Q=17, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00006: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00006: TQ-Probes: [(97.05, 15), (96.99, 16), (95.62, 35)]
INFO [av1an_core::target_quality] chunk 00006: Target Q=15, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00007: P-Rate=4, 472 frames
INFO [av1an_core::target_quality] chunk 00007: TQ-Probes: [(97.67, 15), (96.98, 21), (94.93, 35)]
INFO [av1an_core::target_quality] chunk 00007: Target Q=20, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00012: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00012: TQ-Probes: [(98.16, 15), (97.06, 26), (95.37, 35)]
INFO [av1an_core::target_quality] chunk 00012: Target Q=26, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00008: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00008: TQ-Probes: [(97.19, 35), (96.92, 36), (85.69, 55)]
INFO [av1an_core::target_quality] chunk 00008: Target Q=35, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00002: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00002: TQ-Probes: [(97.27, 15), (97.11, 20), (96.98, 22), (95.92, 35)]
INFO [av1an_core::target_quality] chunk 00002: Target Q=21, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00005: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00005: TQ-Probes: [(97.22, 15), (97.08, 19), (97.00, 20), (96.04, 35)]
INFO [av1an_core::target_quality] chunk 00005: Target Q=19, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00003: P-Rate=4, 472 frames
INFO [av1an_core::target_quality] chunk 00003: TQ-Probes: [(97.17, 15), (97.05, 17), (96.98, 18), (95.44, 35)]
INFO [av1an_core::target_quality] chunk 00003: Target Q=17, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00009: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00009: TQ-Probes: [(97.72, 15), (97.16, 23), (97.11, 24), (95.47, 35)]
INFO [av1an_core::target_quality] chunk 00009: Target Q=24, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00011: P-Rate=4, 473 frames
INFO [av1an_core::target_quality] chunk 00011: TQ-Probes: [(98.62, 15), (97.20, 29), (97.02, 30), (95.91, 35)]
INFO [av1an_core::target_quality] chunk 00011: Target Q=30, VMAF=97.00
INFO [av1an_core::target_quality] chunk 00010: P-Rate=4, 472 frames
INFO [av1an_core::target_quality] chunk 00010: TQ-Probes: [(97.19, 15), (97.05, 17), (96.99, 18), (95.26, 35)]
INFO [av1an_core::target_quality] chunk 00010: Target Q=17, VMAF=97.00

0.4.0

DEBUG [av1an_core::settings] temporary directory: .926b233
INFO [av1an_core::settings] Input: 4096x2160 @ 50.000 fps, YUV422P10LE, HDR
INFO [av1an_core::settings] scenecut: found 1 scene(s) [with extra_splits (500 frames): 14 scene(s)]
DEBUG [av1an_core::broker]  started chunk 00001: 473 frames
DEBUG [av1an_core::broker]  started chunk 00002: 473 frames
DEBUG [av1an_core::broker]  started chunk 00004: 473 frames
DEBUG [av1an_core::broker]  started chunk 00005: 473 frames
DEBUG [av1an_core::broker]  started chunk 00006: 473 frames
DEBUG [av1an_core::broker]  started chunk 00008: 473 frames
DEBUG [av1an_core::broker]  started chunk 00009: 473 frames
DEBUG [av1an_core::broker]  started chunk 00011: 473 frames
DEBUG [av1an_core::broker]  started chunk 00012: 473 frames
DEBUG [av1an_core::broker]  started chunk 00013: 473 frames
DEBUG [av1an_core::broker]  started chunk 00000: 472 frames
DEBUG [av1an_core::broker]  started chunk 00003: 472 frames
DEBUG [av1an_core::broker]  started chunk 00007: 472 frames
DEBUG [av1an_core::broker]  started chunk 00010: 472 frames
DEBUG [av1an_core::target_quality] chunk 00010: P-Rate=1, 472 frames
DEBUG [av1an_core::target_quality] chunk 00010: TQ-Probes: [(96.72, 15), (94.88, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00010: Target Q=15, VMAF=96.72
DEBUG [av1an_core::target_quality] chunk 00000: P-Rate=1, 472 frames
DEBUG [av1an_core::target_quality] chunk 00000: TQ-Probes: [(96.19, 15), (94.65, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00000: Target Q=15, VMAF=96.19
DEBUG [av1an_core::target_quality] chunk 00004: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00004: TQ-Probes: [(96.72, 15), (95.17, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00004: Target Q=15, VMAF=96.72
DEBUG [av1an_core::target_quality] chunk 00005: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00005: TQ-Probes: [(96.84, 15), (95.33, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00005: Target Q=15, VMAF=96.84
DEBUG [av1an_core::target_quality] chunk 00006: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00006: TQ-Probes: [(96.55, 15), (94.38, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00006: Target Q=15, VMAF=96.55
DEBUG [av1an_core::target_quality] chunk 00002: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00002: TQ-Probes: [(96.56, 15), (94.64, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00002: Target Q=15, VMAF=96.56
DEBUG [av1an_core::target_quality] chunk 00013: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00013: TQ-Probes: [(96.15, 15), (92.61, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00013: Target Q=15, VMAF=96.15
DEBUG [av1an_core::target_quality] chunk 00001: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00001: TQ-Probes: [(96.41, 15), (94.32, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00001: Target Q=15, VMAF=96.41
DEBUG [av1an_core::target_quality] chunk 00012: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00012: TQ-Probes: [(96.70, 15), (93.11, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00012: Target Q=15, VMAF=96.70
DEBUG [av1an_core::target_quality] chunk 00011: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00011: TQ-Probes: [(96.79, 15), (94.07, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00011: Target Q=15, VMAF=96.79
DEBUG [av1an_core::target_quality] chunk 00009: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00009: TQ-Probes: [(96.61, 15), (94.94, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00009: Target Q=15, VMAF=96.61
DEBUG [av1an_core::target_quality] chunk 00007: P-Rate=1, 472 frames
DEBUG [av1an_core::target_quality] chunk 00007: TQ-Probes: [(96.29, 15), (93.71, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00007: Target Q=15, VMAF=96.29
DEBUG [av1an_core::target_quality] chunk 00003: P-Rate=1, 472 frames
DEBUG [av1an_core::target_quality] chunk 00003: TQ-Probes: [(96.50, 15), (94.31, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00003: Target Q=15, VMAF=96.50
DEBUG [av1an_core::target_quality] chunk 00008: P-Rate=1, 473 frames
DEBUG [av1an_core::target_quality] chunk 00008: TQ-Probes: [(96.48, 15), (93.88, 35)] Early Skip Low Q
DEBUG [av1an_core::target_quality] chunk 00008: Target Q=15, VMAF=96.48

TLDR:

Is this shift expected with 0.4.0? Should I simply use lower target quality now (maybe 96.5), since my older encoding with 97 where OK quality for me?

aexoden commented 1 year ago

It appears that between 0.3.1 and 0.4.0, the percentile for target quality changed from 25% to 1%, so lowering your target quality value may be the correct option.

doits commented 1 year ago

Thanks for the information. I guess it is this PR: https://github.com/master-of-zen/Av1an/pull/641

I guess I'll just test with different target qualities then again to find the optimal for me.

BlueSwordM commented 1 year ago

Basically, there are 2 things that changed:

  1. We went from 4 frame subsampling to no frame subsampling. That makes VMAF analysis slower(not as much of a concern these days), but much more accurate.

  2. We went from scoring the 25% worst frames to the 1% worst frames. I'd have personally set it to 5%, but MOZ wanted it that way, so I don't care :)