Use -filmgrain 0 when running VMAF command for AV1 sources

[DesertCookie]

AV1 comes with built-in grain synthesis that allows for major bitrate reductions by sythesising, removing, and then re-adding grain on playback. However, this grain is only similar in nauture to the original, never identical. VMAF and other scoring systems will pick this up as difference to the original.

This can be somewhat alleviated by running VMAF with -filmgrain 0 which disables the addition on decoding and usually results in a more accurate (higher) score. Could this option be added to the next release of FFMetrics, possibly as a toggle or similar control?

[fifonik]

Your suggestion sounds reasonable, however I do not see the -filmgrain option in ffmpeg vmaf manual: http://ffmpeg.org/ffmpeg-filters.html#libvmaf

[DesertCookie]

I am unsure of how new this might be. I've got this tip from here. I found it in the regular docs; does VMAF not simply decode using FFmpeg and thus also benefit from ordinary decode parameters?

Edit: Maybe should have clarified this is an FFmpeg option, not a VMAF option in my OG request.

[fifonik]

Ah, I see, this is av1-decoder option.

[fifonik]

I will try to add it (as config configurable option for now as I do not see any reason to have the grain added when calculating comparison metrics, may be I will change my mind and add it as an UI option in further). However, will need to find out how yo use it with ffmpeg first, and to do this I will need to find a way to encode to av1 %-)

[DesertCookie]

Just a heads up: It seems like filmgrain 0 is going to get deprecated soon, someone pointed out here.

There's lots of AV1 test clips available on the internet; most popular 4K and up YouTube videos are in AV1 now. And for easy AV1 encoding using FFmpeg I suggest FastFlix.

[fifonik]

I've already encoded to AV1 and played with -filmgrain 0 option (noticed the deprecation message you mentioned). Could not understand how to use the new "export" option. Thanks for the message, I will try to use -export_side_data film_grain

[fifonik]

I prepared uncompressed video with the same encoded as av1 (I encoded from VP19 using Voukoder, but it does not really matter, the fact is -- I got uncompressed avi + av1 mp4).

Then I compared it a few times using ffmpeg 4.4 (gyan build):

C:\Vegas\AV1-filmgrain>ffmpeg
ffmpeg version 4.4-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

I ended up with 3 exactly the same VMAF score values:

C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d @ 000002a19f918440] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Voukoder (VEGAS)
  Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
  Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
  Metadata:
    TCOD            : 0
    TCDO            : 212333333
  Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
  Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d @ 000002a19fad02c0] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
  Stream #0:0 (libdav1d) -> settb (graph 0)
  Stream #1:0 (rawvideo) -> settb (graph 0)
  libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 wrapped_avframe
  Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 pcm_s16le
frame=  637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed=0.906x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 000002a1a1368cc0] VMAF score: 86.563790

C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -filmgrain 0 -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d decoder @ 000002e0187aad80] The "filmgrain" option is deprecated: Apply Film Grain
[libdav1d @ 000002e0187994c0] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Voukoder (VEGAS)
  Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
  Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
  Metadata:
    TCOD            : 0
    TCDO            : 212333333
  Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
  Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d decoder @ 000002e01a6ba980] The "filmgrain" option is deprecated: Apply Film Grain
[libdav1d @ 000002e01a8e1240] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
  Stream #0:0 (libdav1d) -> settb (graph 0)
  Stream #1:0 (rawvideo) -> settb (graph 0)
  libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 wrapped_avframe
  Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 pcm_s16le
frame=  637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed=0.898x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 000002e01a6ad880] VMAF score: 86.563790

C:\Vegas\AV1-filmgrain>ffmpeg.exe -hide_banner -nostdin -export_side_data film_grain -i av1-good-CQ50.mp4 -i uncompressed.avi -lavfi [0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]libvmaf=eof_action=endall:n_threads=15:pool=Mean:model_path='vmaf-models/vmaf_v0.6.1neg.json' -f null -
[libdav1d @ 0000029ecf0694c0] libdav1d 0.8.2-2-g6c6d25d
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'av1-good-CQ50.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Voukoder (VEGAS)
  Duration: 00:00:21.27, start: 0.000000, bitrate: 16464 kb/s
  Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 16480 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 7 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Input #1, avi, from 'uncompressed.avi':
  Metadata:
    TCOD            : 0
    TCDO            : 212333333
  Duration: 00:00:21.23, start: 0.000000, bitrate: 2654394 kb/s
  Stream #1:0: Video: rawvideo, bgr24, 2560x1440, 2658381 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
[libdav1d @ 0000029ecf551040] libdav1d 0.8.2-2-g6c6d25d
Stream mapping:
  Stream #0:0 (libdav1d) -> settb (graph 0)
  Stream #1:0 (rawvideo) -> settb (graph 0)
  libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
←[35mlibvmaf←[0m ←[32mINFO←[0m `compute_vmaf()` is deprecated and will be removed in a future libvmaf version
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 wrapped_avframe
  Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 pcm_s16le
frame=  637 fps= 27 q=-0.0 Lsize=N/A time=00:00:21.24 bitrate=N/A speed= 0.9x
video:333kB audio:11952kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libvmaf @ 0000029ed09fec80] VMAF score: 86.563790

Coult you provide your commands when you received different values using the -filmgrain 0 option please?

P.S I've also tried ffmpeg 5.0.1 (gyan) and ffmpeg 5.0 (BtBN) with the same outcome.

[DesertCookie]

I have not experienced different values as I've always used FFMetrics for VMAF. Maybe the person that commented to me on Reddit can add more insight as he clearly seemed knowledgable about this.

[fifonik]

I tried to follow his comments on reddit. Unfortunately, my knowledge in ffmpeg/AV1 not allow me to understand everything and so to use the information he provided.

Sorry, I'm unable to do anything right now.

P.S. It is possible that AV1 must contain some flag that is turning on the film grain during decoding. But the AV1 I rendered myself do not have the flag. Unfortunately, I do not know how to check it. I do not see anything related to film grain in the encoder I have.

[DesertCookie]

Alright, maybe there's something going on where FFmpeg, by default, doesn't synthesise the grain on playback (some players don't as it's very resource intensive). Thank you for trying!