Open DesertCookie opened 1 year ago
Your suggestion sounds reasonable, however I do not see the -filmgrain option in ffmpeg vmaf manual: http://ffmpeg.org/ffmpeg-filters.html#libvmaf
I am unsure of how new this might be. I've got this tip from here. I found it in the regular docs; does VMAF not simply decode using FFmpeg and thus also benefit from ordinary decode parameters?
Edit: Maybe should have clarified this is an FFmpeg option, not a VMAF option in my OG request.
Ah, I see, this is av1-decoder option.
I will try to add it (as config configurable option for now as I do not see any reason to have the grain added when calculating comparison metrics, may be I will change my mind and add it as an UI option in further). However, will need to find out how yo use it with ffmpeg first, and to do this I will need to find a way to encode to av1 %-)
I've already encoded to AV1 and played with -filmgrain 0
option (noticed the deprecation message you mentioned).
Could not understand how to use the new "export" option. Thanks for the message, I will try to use -export_side_data film_grain
I prepared uncompressed video with the same encoded as av1 (I encoded from VP19 using Voukoder, but it does not really matter, the fact is -- I got uncompressed avi + av1 mp4).
Then I compared it a few times using ffmpeg 4.4 (gyan build):
C:\Vegas\AV1-filmgrain>ffmpeg
ffmpeg version 4.4-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Use -h to get full help or, even better, run 'man ffmpeg'
I ended up with 3 exactly the same VMAF score values:
C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d @ 000002a19f918440] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Voukoder (VEGAS)
Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
Metadata:
TCOD : 0
TCDO : 212333333
Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d @ 000002a19fad02c0] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
Stream #0:0 (libdav1d) -> settb (graph 0)
Stream #1:0 (rawvideo) -> settb (graph 0)
libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Lavf58.76.100
Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
Metadata:
encoder : Lavc58.134.100 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
encoder : Lavc58.134.100 pcm_s16le
frame= 637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed=0.906x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 000002a1a1368cc0] VMAF score: 86.563790
C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -filmgrain 0 -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d decoder @ 000002e0187aad80] The "filmgrain" option is deprecated: Apply Film Grain
[libdav1d @ 000002e0187994c0] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Voukoder (VEGAS)
Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
Metadata:
TCOD : 0
TCDO : 212333333
Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d decoder @ 000002e01a6ba980] The "filmgrain" option is deprecated: Apply Film Grain
[libdav1d @ 000002e01a8e1240] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
Stream #0:0 (libdav1d) -> settb (graph 0)
Stream #1:0 (rawvideo) -> settb (graph 0)
libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Lavf58.76.100
Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
Metadata:
encoder : Lavc58.134.100 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
encoder : Lavc58.134.100 pcm_s16le
frame= 637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed=0.898x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 000002e01a6ad880] VMAF score: 86.563790
C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -export_side_data film_grain -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d @ 0000029ecf0694c0] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Voukoder (VEGAS)
Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
Metadata:
TCOD : 0
TCDO : 212333333
Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d @ 0000029ecf551040] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
Stream #0:0 (libdav1d) -> settb (graph 0)
Stream #1:0 (rawvideo) -> settb (graph 0)
libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomav01iso2mp41
encoder : Lavf58.76.100
Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
Metadata:
encoder : Lavc58.134.100 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
encoder : Lavc58.134.100 pcm_s16le
frame= 637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed= 0.9x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 0000029ed09fec80] VMAF score: 86.563790
Coult you provide your commands when you received different values using the -filmgrain 0 option please?
P.S I've also tried ffmpeg 5.0.1 (gyan) and ffmpeg 5.0 (BtBN) with the same outcome.
I have not experienced different values as I've always used FFMetrics for VMAF. Maybe the person that commented to me on Reddit can add more insight as he clearly seemed knowledgable about this.
I tried to follow his comments on reddit. Unfortunately, my knowledge in ffmpeg/AV1 not allow me to understand everything and so to use the information he provided.
Sorry, I'm unable to do anything right now.
P.S. It is possible that AV1 must contain some flag that is turning on the film grain during decoding. But the AV1 I rendered myself do not have the flag. Unfortunately, I do not know how to check it. I do not see anything related to film grain in the encoder I have.
Alright, maybe there's something going on where FFmpeg, by default, doesn't synthesise the grain on playback (some players don't as it's very resource intensive). Thank you for trying!
AV1 comes with built-in grain synthesis that allows for major bitrate reductions by sythesising, removing, and then re-adding grain on playback. However, this grain is only similar in nauture to the original, never identical. VMAF and other scoring systems will pick this up as difference to the original.
This can be somewhat alleviated by running VMAF with
-filmgrain 0
which disables the addition on decoding and usually results in a more accurate (higher) score. Could this option be added to the next release of FFMetrics, possibly as a toggle or similar control?