[ffmpeg+vmaf] could not parse model config for ffmpeg.

[myselfzhangji]

Hi, I got an error "could not parse model config" as following: [Parsed_libvmaf_0 @ 000002af16615a40] could not parse model config: path=D:/ffmpeg/bin/model/vmaf_b_v0.6.3.json [AVFilterGraph @ 000002af165fce80] Error initializing filters Failed to set value 'libvmaf=model='path=D\:/ffmpeg/bin/model/vmaf_b_v0.6.3.json'' for option 'lavfi': Invalid argument Error parsing global options: Invalid argument

My test cmd is as following on Windows: ffmpeg -i output.mp4 -i test.mp4 -lavfi "libvmaf=model='path=D\\:/ffmpeg/bin/model/vmaf_b_v0.6.3.json'" -f null -

And my ffmpeg version is "7.0.2-full_build-www.gyan.dev" for Windows. $ ffmpeg ffmpeg version 7.0.2-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers

I have such model file under my windows path "D:/ffmpeg/bin/model/vmaf_b_v0.6.3.json", can someone share suggestions to me, sorry for asking this silly question.

BTW, if I doesn't specify any model for libvmaf, it works, and I think I am using the default model "vmaf_v0.6.1.json" to calculate vmaf score, is my understanding right? ffmpeg -i output.mp4 -i test.mp4 -lavfi libvmaf -f null -

And the log info on console is as following, with "libvmaf:default -> Stream #0:0 (wrapped_avframe)"

[nilfm99]

Hi @myselfzhangji, thanks for raising this issue. I am able to reproduce locally that the model vmaf_b_v0.6.3 in particular is not able to load when called from ffmpeg:

ffmpeg -i dis5.y4m -i ref5.y4m -lavfi "libvmaf=model=version=vmaf_v0.6.1" -f null -
> loads correctly

ffmpeg -i dis5.y4m -i ref5.y4m -lavfi "libvmaf=model=version=vmaf_v0.6.1neg" -f null -
> loads correctly

ffmpeg -i dis5.y4m -i ref5.y4m -lavfi "libvmaf=model=version=vmaf_b_v0.6.3" -f null -
> [Parsed_libvmaf_0 @ 0x6000034dc000] could not load libvmaf model with version: vmaf_b_v0.6.3

However, I am able to load this model from the vmaf executable:

debug/tools/vmaf -r ~/inputs/ref5.y4m -d ~/inputs/dis5.y4m --frame_cnt 1 --model=version=vmaf_b_v0.6.3

VMAF version d95b69e0
1 frame  ⢀⠀ 0.00 FPS
vmaf_b_v0.6.3: 7.288460
vmaf_b_v0.6.3: 7.637194, ci.p95: [0.000000, 14.110062], stddev: 4.583206

I did some more digging and found this:

// check for model_collection before failing
// this is implicit because the `--model` option could take either
// a model or model_collection

which is not replicated in the ffmpeg filter, and this is why it works only on the vmaf executable. cc @kylophone for visibility.

Just to make sure that my hypothesis is correct, if you try replacing this model with a non-collection model (for example, vmaf_v0.6.1neg.json), does the error persist?

Until a fix is rolled out, I would suggest using non-collection models, or using the vmaf executable (although you'll need to decode first, which you can do with ffmpeg). Apologies for the inconvenience.

[myselfzhangji]

Hi @nilfm99, Really thanks for your comments. Yes, you are right, as your suggestion, both below 2 cmd works on my side.

 ffmpeg -i output.mp4 -i test.mp4 -lavfi "libvmaf=model=version=vmaf_v0.6.1'" -f null -
 ffmpeg -i output.mp4 -i test.mp4 -lavfi "libvmaf=model=version=vmaf_v0.6.1neg'" -f null -

However, I think with above cmd, it is calculating VMAF score with models built-in with this FFmpeg, is my understanding correct? If I want to specify the specific trained model with libvmaf filters "model=path" as below, how should I use such command? can you give me some sugestions about such kind of use case?

ffmpeg -i output.mp4 -i test.mp4 -lavfi "libvmaf=model='path=D\\:/ffmpeg/bin/model/vmaf_b_v0.6.3.json'" -f null -

[nilfm99]

Hi @myselfzhangji,

Correct, when you specify a model using a version, it takes it from the list of built-in models. If you want to specify a model by path, you should be able to do it with the command you specified, with the caveat that (due to the reasons I mentioned in my other comment) ModelCollection is not supported from the ffmpeg interface, so you'll have to supply a single Model instead.

To see the difference, take a look at this example:

• single model
• model collection

See how in the model collection, there is an extra layer where the key "0" corresponds with a single model, "1" corresponds to another one, etc. Those are currently not supported from ffmpeg. If you have any further problems please let me know how you obtained the model, I'll take a closer look.

[myselfzhangji]

Hi @nilfm99,

There is still such issue even I use "single model" as in below cmd:

ffmpeg -i output.mp4 -i test.mp4 -lavfi "libvmaf=model='path=D\:/ffmpeg/bin/model/vmaf_v0.6.1.json'" -f null -

vmaf_v0.6.1.json is I donwnload from vmaf github : https://github.com/Netflix/vmaf/tree/master/model. Please see its content from vmaf_v0.6.1.json

It seems there is unexpected issue. From the output log when calculating VMAF score, I can see that my ffmpeg is built-in with libvmaf version "3.0.0", please see the log info from vmaf.log

And the donwloaded vmaf is also "3.0.0", I don't understand why there is such issue "could not parse model config: path=D:/ffmpeg/bin/model/vmaf_v0.6.1.json", please see below

And I have such mode in the path "D:/ffmpeg/bin/model/"

BTW, I try to study the vmaf source code in my ubuntu virtual machine, and I "grep" this error, I even don't find why this error info is from, this is also unreasonable. Is there any mismatch for this libvmaf built-in with the ffmpeg version?

[nilfm99]

Hi, thanks for providing more details. Your issue is a different one then, since the issue I described is limited to the "model collection" models, and vmaf_v0.6.1.json is not one of them.

To answer your last question first, the error is coming from this line in ffmpeg. When you call ffmpeg, that file is responsible for interfacing with VMAF, but it is not part of VMAF itself. This is why you couldn't find the error when you grepped VMAF.

That also means that the error is happening before any VMAF model parsing function is called. Most likely, the error is in this function, which simplifies things because that's just pure C code to parse the option you supplied.

The only way I could reproduce your issue is if my path includes a colon, like yours does. You can see from the log that the string passed to the delimited_dict_parse function is not escaped, something in ffmpeg probably consumed the backslash. The same happens if you double the backslash. By trial and error, I arrived at five backslashes, which works for me (well, it fails because it can't find the model, but it should find it in your machine).

ffmpeg -i dis5.y4m -i ref5.y4m -lavfi "libvmaf=model='path=D\\\\\:/ffmpeg/bin/model/vmaf_v0.6.1.json'" -f null - 

libvmaf ERROR could not read model from path: "D:/ffmpeg/bin/model/vmaf_v0.6.1.json"
[Parsed_libvmaf_0 @ 0x600000f40000] could not load libvmaf model with path: D:/ffmpeg/bin/model/vmaf_v0.6.1.json
[AVFilterGraph @ 0x60000134c2a0] Error initializing filters
Failed to set value 'libvmaf=model='path=D\\\:/ffmpeg/bin/model/vmaf_v0.6.1.json'' for option 'lavfi': Invalid argument
Error parsing global options: Invalid argument

See how the error message is different. I believe the shell consumes two backslashes and ffmpeg consumes another two, though I haven't looked at it in depth. Please try this and a few more amounts of backslashes, and let me know if nothing works. Escaping in ffmpeg is generally painful and it is not limited to the libvmaf filter.

[myselfzhangji]

Hi @nilfm99 ,

Thanks a lot for the comment. It is the issue that ffmpeg handles the specified path. I would switch to my ubuntu to run ffmpeg + vmaf.

Really thanks a lot for your help.

[nilfm99]

No problem @myselfzhangji, glad you resolved it. I'll close this issue, please reopen it if you have further issues with this.