It's the path issue for new custom ffmpeg builds with vmaf.

[ls-milkyway]

Hi!, vmaf is not detected in custom compiles ,,,,because ffmpeg command is not using proper absolute path: Error initializing filter 'libvmaf' with args 'F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json' I am attaching log in debug mode FFMetrics.log

Using absolute path is tricky ...plz check this

I guess a simple relative path vmaf-models/vmaf_v0.6.1.json or vmaf-models/vmaf_4k_v0.6.1.json will work as the folder is present in the app . Hence, what parameters should I pass to "-vmaf-model=filename" for running ffmetrics with command line options?? I have tried -vmaf-model=vmaf-models/vmaf_v0.6.1.json but it's not working....vmaf is still not detected by ffmetrics?

[fifonik]

It looks like you have not turned on option to save ffmpeg commands to log in program menu. When you do so, you will have in log file whole 'ffmpeg ...options...' commands (easier to copy/past and check if they work).

However, from what I see, path to model looks correct (believe me, I've checked the article about paths to ffmpeg you provided many times). Do you use vmaf models from the ffmetrics archive? If you downloaded the models yourself, could you attach your F:\Audio Video\FFMetrics.1.0.0\vmaf-models\vmaf_4k_v0.6.1.json file please?

Have you already tried to put gyan's static ffmpeg.exe aside of ffmetrics.exe?

I guess a simple relative path vmaf-models/vmaf_v0.6.1.json or vmaf-models/vmaf_4k_v0.6.1.json will work

Unfortunately, no. ffmpeg could be somewhere in %PATH%, so full file spec must be provided.

what parameters should I pass to "-vmaf-model=filename"

If I remember correctly, you should only specify model name without extension: ffmetrics -vmaf-model=vmaf_4k_v0.6.1 In any case, specifying model manually as command line parameter will not help with your issue. ffmetrics only use the option to select model in dropdown. That's it.

P.S. I hope I will be able to investigate how ffmetrics work with new ffmpeg version over the weekend.

[ls-milkyway]

It looks like you have not turned on option to save ffmpeg commands to log in program menu. When you do so, you will have in log file whole 'ffmpeg ...options...' commands (easier to copy/past and check if they work).

Well, i am not saying that my compiled FFmpeg is not working ...it is in fact working & calculating psnr & ssim .....but not showing vmaf option....because log says WARNING: VMAF metric initialization failed So, I need only debug logs (attached earlier) to why vmaf is not being detected?

Do you use vmaf models from the ffmetrics archive?

Yes i am using models that came with ffmetrics ...i.e. inside folder: vmaf-models

Have you already tried to put gyan's static ffmpeg.exe aside of ffmetrics.exe?

Yes I am aware that both BtBN & gyan's release work well with FFmetrics....but that kills the purpose of my custom build....i mean previously I was using 2 to 3 copies of ffmpegs....one non-free (with libfdk-aac) ...other one with vmaf (btbn).....and another one for vvc .....hence took a long time to understand & compile all 3 in one & built my personal ffmpeg .... FYI: vmaf via cmd is working:

ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model=path=models/vmaf_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

But not with FFmetrics (vmaf not available/highlighted) ....as i need graph output....I will be highly obliged if you [link removed] (it will run on any pc as it is built with cpu generic) & give it a try.

Do not share it as it will void non-free ffmpeg license ...fyi: the folder with ffmpeg, ffplay, ffprobe was scanned with KIS latest updates....and compressed with winrar ....is absolutely safe.

[fifonik]

Well, i am not saying that my compiled FFmpeg is not working

Sorry, I meant if VMAF detection and VMAF calculation works with official builds. If it does not work as well, then the issue is definitely not with the custom build.

log says WARNING: VMAF metric initialization failed So, I need only debug logs (attached earlier) to why vmaf is not being detected?

FFMetrics not only executing ffmpeg while calculating metrics. It is also calling it on startup a few times to detect available features (such as: to get version info, to check if ffmpeg is compiled with vmaf, to check what vmaf model types supported -- pkl or json). Then it is calling ffmpeg to get media info and ref's thumbnail. When you turn on the menu option 'save ffmpeg commands to log' and restart the program, you will see ALL these auxiliary commands in log as well. So it would be easy to copy them and check in command line with different ffmpegs.

FYI: vmaf via cmd is working: ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model=path=models/vmaf_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

You can try to supply full path to vmaf model here: ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model=path=F\\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

Then you can try to supply deprecated format that is used by ffmetrics: ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model_path=F\\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

Unfortunately, I do not know if the ffmpeg will try to load external vmaf models if it was compiled with build-in vmaf models.

P.S. I downloaded your build and removed link in your message. Feel free to delete the archive.

[ls-milkyway]

It is also calling it on startup a few times to detect available features (such as: to get version info, to check if ffmpeg is compiled with vmaf, to check what vmaf model types supported -- pkl or json).

Hmm...my custom build ffmpeg has peculiar version: N-106631-ga38e3eb503-g8b717a4044+1.....I hope FFmetrics is able to parse the value correctly...... Since I have built ffmpeg with latest master branches of all components (to avoid bugs/vulnerabilities) ...hence: 1) It requires json file only 2) Copied the BTBN parameters for compiling vmaf i.e. -Denable_float=true -Denable_tests=false -Denable_docs=false --buildtype=release --default-library=static -Dbuilt_in_models=true -Denable_avx512=true 3) For your reference: the built in models in vmaf are here.

If FFmetrics is looking for any other model files (say .pkl) to enable vmaf ...then I strongly advice you to remove it ...& check only json files....as -Dbuilt_in_models=true is the default now in vmaf compilation......to make FFmetrics compliant to future ffmpeg releases. Both btbn & gyan may not be using latest master builds.

Since he is also using -Dbuilt_in_models=true & is getting detected by FFmetrics ....so I believe built in models is not an issue.

You can try to supply full path to vmaf model here: ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model=path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

My ffmpeg refuses to take any full path with drive letter ....it is only working when provided with internal folder path....like model=path=models/vmaf_v0.6.1.json OR model=path=vmaf-models/vmaf_4kv0.6.1.json May be they have compiled their releases on the docker....and me on a drive with static path .....so it may be messing up.....

I have tried various path options both via command & via FFmetrics.conf by using model_path={{model_path}} as model_path=F\\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json but it's not working ....I also wanted to check the source code of the FFmetrics for troubleshooting ...but it is yet to be published.

I also wish to have MS-SSIM included in FFmetrics ....already available with vmafossexec ....

[fifonik]

my custom build ffmpeg has peculiar version: N-106631-ga38e3eb503-g8b717a4044+1 .....I hope FFmetrics is able to parse the value correctly

As all BTBN's nightly builds. FFMetrics does not make any predictions on available ffmpeg's features based on version "number". It is only shows ffmpeg version in status bar.

My ffmpeg refuses to take any full path with drive letter

This is strange. I checked vmaf's sources and it should allow you to use path. Unfortunately, if it does not for some reason -- you cannot use it with FFMetrics at this stage then :( At this moment I'm making changes in FFMetrics to make it possible.

[fifonik]

Could you please run from command line (try it for your and BtBN/gyan builds, to be sure that these builds works with these models):

ffmpeg -i output.mkv -i ready.mkv -filter_complex libvmaf='model_path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json' -f null -

Notice:

• No double quotes around -filter_complex/-lavfi options
• Single quotes around libvmaf options
• Single backslash after drive letter in model path

I hope this will work.

Double quotes can be used around libvmaf options, but in this case you have to use double backslash after drive letter:

ffmpeg -i output.mkv -i ready.mkv -filter_complex libvmaf="model_path=F\\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

If you use no quotes or single quotes for libvmaf options, you can also put double quotes around the whole -filter_complex/-lavfi options. libvmaf options still must be quoted in case you have space in model/log path.

ffmpeg -i output.mkv -i ready.mkv -filter_complex "libvmaf='model_path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json'" -f null -

[ls-milkyway]

ffmpeg -i output.mkv -i ready.mkv -filter_complex libvmaf='model_path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json' -f null -

Fails on both my-build as well with btbn & gyan's with output:

[libvmaf @ 00000000001cfb00] The "model_path" option is deprecated: use model='path=...'. ←[35mlibvmaf←[0m ←[31mERROR←[0m could not read model from path: "F:/Audio" [Parsed_libvmaf_0 @ 0000000002a67e40] problem loading model file: F:/Audio [AVFilterGraph @ 0000000000115180] Error initializing filter 'libvmaf' with args 'model_path=F\:/Audio' Error initializing complex filters. Invalid argument

Rest 2 commands also fails on my build with output: (also fails in other 2 as filter complex does not uses frame size.....hence output's frame size is != the ready's frame size)

`[libvmaf @ 000000000058c040] The "model_path" option is deprecated: use model='path=...'. ←[35mlibvmaf←[0m ←[33mWARNING←[0m no such built-in model: "vmaf_v0.6.1" [Parsed_libvmaf_0 @ 000000000058bf40] could not load libvmaf model with version: vmaf_v0.6.1 [AVFilterGraph @ 0000000000585180] Error initializing filter 'libvmaf' with args 'model_path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=V MAF_out.json:n_threads=2:log_fmt=json' Error initializing complex filters. Invalid argument

[libvmaf @ 00000000004ac040] The "model_path" option is deprecated: use model='path=...'. ←[35mlibvmaf←[0m ←[33mWARNING←[0m no such built-in model: "vmaf_v0.6.1" [Parsed_libvmaf_0 @ 00000000004abf40] could not load libvmaf model with version: vmaf_v0.6.1 [AVFilterGraph @ 00000000004a5180] Error initializing filter 'libvmaf' with args 'model_path=F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json:log_path=V MAF_out.json:n_threads=2:log_fmt=json' Error initializing complex filters. Invalid argument`

Hmm, I understand you are trying to get a command which works on all builds.....& FYI the following command (using relative path works on all 3):

ffmpeg -i output.mkv -i ready.mkv -filter_complex "[0:v]scale=3840:1608:flags=bicubic[main];[main][1:v]libvmaf=model=path=vmaf-models/vmaf_v0.6.1.json:log_path=VMAF_out.json:n_threads=2:log_fmt=json" -f null -

but I am not able to understand why are you interested in absolute paths? (sorry not much into development ....but I guess ...if relative paths can work to detect vmaf by ffmetrics ....then why not to use it)?

Moreover folder "vmaf-models" will always be inside say...FFMetrics.1.0.0....as the app does not use an installer with custom location..i.e..... it's a zip file with pwd path always relative to the folder "vmaf-models" ...unless user changes it ...so why not avoid full path like "F\:/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_4k_v0.6.1.json"? Unless there are chances of bugs or some discrepancies occurring due to the relative path??

[fifonik]

Path to the model is only one part of the story. FFMetrics also specifying path to log (while calculating VMAF metric) and path to image (while extracting bad frames). So it will be absolute paths there anyway. So the program will fail just a bit later. This is why I'm trying to understand why it fails in your system while works fine in mine.

[fifonik]

Could you please download the archive and unpack it into C:\Temp Then put gyan's ffmpeg.exe into the C:\Temp\ff and run the ff.bat.

It works with no issues here, see included screenshot.png (Note: I'm using deprecated format model_path=... instead of mode=path=... that you use in your examples).

Could you check if this works for you and provide screenshot please?

[ls-milkyway]

Could you check if this works for you and provide screenshot please?

Gyan's Output:

BtBN's Output:

My-Build's Output:

[fifonik]

I briefly checked ffmpeg/libvmaf source and only see the error message you have in vf_libvmaf.c (line 333) where libvmaf is processing model options in new format if 'version' is specified (model=version=...).

FFMetrics and the ff.bat use deprecated format (model_path=...) so different path of code should be followed and error if system cannot load model file should be different: "problem loading model file" (line 461).

I do not have enough knowledge about ffmpeg/libvmaf internals to further investigate the issue. Sorry.

However, I am making changes in FFMetrics so it should be possible to use ffmpeg's "built-in" vmaf models in future. No ETA at this stage as I have not decided yet how to implement some things (get/define a list of "built-in" models).

P.S. I've only checked ffmpeg/libvmaf master. It is also possible that in version you compiled there were some issues that is now resolved. You may try to re-build your ffmpeg binary from the most recent source.

[ls-milkyway]

FFMetrics and the ff.bat use deprecated format (model_path=...) so different path of code should be followed and error if system cannot load model file should be different: "problem loading model file" (line 461).

Ok........I have the working command with the absolute path!!.....but I cannot understand why it is working? .....may be you can:

Command: ffmpeg.exe -f lavfi -i color=c=black:s=640x480:d=1 -filter_complex "[0:v][0:v]libvmaf=model='path=c\\\:/Temp/Ff/Audio Video/FFMetrics.1.0.0/vmaf-models/vmaf_v0.6.1.json'" -f null -

Screenshot:

I guess ....I did not look deep in to the changed ffmpeg / vmaf docs about model=path= command and windows path. FFmpeg docs are very cryptic in nature....plus I have a hint that there is also a direct command to inbuilt model in new FFmpeg:

For eg. (Not working but it is close to something like this) ffmpeg.exe -f lavfi -i color=c=black:s=640x480:d=1 -filter_complex "[0:v][0:v]libvmaf=model=version=vmaf_v0.6.1" -f null -

Thereby eliminating the need for any model file or folder.

[fifonik]

Thanks for the information. Unfortunately, I do not know. I will "play" with quoting paths only for the deprecated option. May be this will do the trick as well.

As you copied build options from BtBN's code, I think the main question here is: Why the most recent BtBN's build works with no issues, while your build does not?

[ls-milkyway]

Why the most recent BtBN's build works with no issues, while your build does not?

Well, I was just using compiling parameters ....they just add or reduce features in Vmaf...thinking that some features may be messing with paths ...but real story is quiet different....

The quick answer (needs more time to dwell deep) is: BtbN is using a cross-compiler in Linux to build Windows exe....whereas I am building on Windows....and you know windows and Unix path interoperability sucks ....

The Vmaf git is full of problems related to path in windows builds......there is nothing they can do ...The easiest way to solve this would be to have a way for ffmpeg to handle windows paths inside itself, but that's not going to be easy nor happen. The second way is to handle path conversion inside libvmaf, but that will have the problem of cygpath unless we can create a portable way to convert paths.

Moreover, both FFmpeg & Vmaf keep on updating ....without proper & clear documentation.

[fifonik]

Thanks for the explanation.

In the upcoming version user will have more control over lavfi/libvmaf options.

[fifonik]

I've just released version 1.2.0 beta 1

Using this version you should be able to specify parameters that are used for:

1. checking vmaf models availability (this was not available in early versions at all)
2. vmaf calculation (this was improved, so now FFMetrics can substitute relative path to model)

By default nothing changed and you will have the same error. Sorry.

However, you can create FFMetrics.conf (based on provided FFMetrics.conf.exampe) and specify options that works for you. To do so you will need to provide templates for 'lavfi' and 'libvmaf' options. You can put static values there (n_threads=2) or use {{name}}-placeholders (n_threads={{n_threads}}) that FFMetrics will substitute run-time.

At the time of writing the following placeholders supported: {{ref}} -- absolute path to reference file {{main}} -- absolute path to distorted file {{n_threads}} -- a number of logical CPUs minus 1 {{log_fmt}} -- "json" {{log_path}} -- absolute path to log (note: it is not possible to substitute relative path as for model's path) {{model_path}} -- absolute path to selected vmaf model {{model_absolute_path}} -- absolute path to selected vmaf model (the same as above) {{model_relative_path}} -- relative path to selected vmaf model {{model_version}} -- selected vmaf model version (lowercased file name without path and extension). Can be used for working with in-built vmaf models (we can discuss this option later, not yet). {{pool}} -- pooling method selected in UI {{phone_model}} -- '1' if "Phone" checkbox ticked in UI {{enable_transform}} -- 'true' if "Phone" checkbox ticked in UI {{filter}} -- string from "Filter" model option.

So you should be able use libvmaf new format like this: "Filter": "libvmaf=model=path={{model_relative_path}}..."

Just rename FFMetrics.conf.example to FFMetrics.conf and replace model_path={{model_path}} with model_path={{model_relative_path}}. This should allow FFMetrics to detect that VMAF is supported by FFMpeg, but I bet will fail anyway on VMAF calculation becasue VMAF log still be an absolute path with single backslash colon escaping. As of now, you can create a folder on the same drive and replace log_path={{log_path}} with log_path=relative/path/to/vmaf.log in the conf. To deal with it I need to find a way to detect how ffmpeg parsing command line params (see my other comment below) or to introduce a new option (doint it now).

In case of issues:

• tick option "Write FFMpeg commands to log"
• exit FFMetrics
• delete log
• run FFMetrics.exe --log-level=debug
• calculate VMAF (The log will be usable even VMAF is not available)
• exit FFMetrics
• provide log and FFMetrics.conf

[fifonik]

The quick answer (needs more time to dwell deep) is: BtbN is using a cross-compiler in Linux to build Windows exe....whereas I am building on Windows....and you know windows and Unix path interoperability sucks ....

Do you have any ideas how I can try to guess based on "configuration" ffmpeg returned how the executable was buid? So, if I detect that it was build on windows, I can escape colon in filespec with triple back slashes, while when it was build on *nix -- single backslash. Just curious, how gyan is building it? It shows "Build by MSYS2 project" as for your build.

[fifonik]

Published version 1.2.1 beta1 Now you can specify escape characters that will be used for escaping ':' in file specs (both model and log) that substituted into ffmpeg libvmaf params (see FFMetrics.conf.example).

Note: The conf is in json, every single '\'-char should be escaped with another '\'. As a result, in your case you will need to put in conf something like this: "Escape": "\\\\\\" %-) You can check how FFMetrics actually build the ffmpeg commands turning ON option 'Write FFMpeg commands into log'

With these changes, I believe, you should be able to use your ffmpeg.exe with FFMetrics.

Let me know if you have any questions.

[ls-milkyway]

With these changes, I believe, you should be able to use your ffmpeg.exe with FFMetrics.

Thanks!