Add support for VMAF Cuda processing?

[teb]

hi, would it be possible to add CUDA processing offloading for VMAF calculation? As well as parallelize processing of multiple files at the same time?

https://github.com/Netflix/vmaf/pull/1080

[fifonik]

Is it supported by ffmpeg? If 'yes' you can simply adjust command for calculating VMAF in FFMetrics.conf (see FFMetrics.conf.example)

[teb]

Seems to be soon via ffmpeg-next.. gonna be interessting to test when someone is able to build it for windows

[maxpain]

Any updates?

[fifonik]

What kind of update you expecting? It is possible to adjust options passed to ffmpeg by editing FFMetrics.conf. So you can add an option that will force ffmpeg to use CUDA (if it is supported).

[guiweber]

Since this app aims to allow using vmaf without having to deal with the command line, it would be nice to have this option implemented in the GUI :)

[tormento]

What kind of update you expecting? It is possible to adjust options passed to ffmpeg by editing FFMetrics.conf. So you can add an option that will force ffmpeg to use CUDA (if it is supported).

What exactly should we add and where?

[fifonik]

I'm sorry, I did not get what you asking.

[tormento]

Sorry, some part of my reply went in the quote.

Now that I edited it should be clear.

[fifonik]

I saw the question inside quotes and no, it is still not clear. Did you ask what exactly you need to add into config file to force ffmpeg to use CUDA while processing file?

[tormento]

I saw the question inside quotes and no, it is still not clear. Did you ask what exactly you need to add into config file to force ffmpeg to use CUDA while processing file?

Yes and also the other parameters that we can introduce. Unfortunately, to my eyes, the syntax of that configuration file is a bit obscure.

[guiweber]

I saw the question inside quotes and no, it is still not clear. Did you ask what exactly you need to add into config file to force ffmpeg to use CUDA while processing file?

Yes and also the other parameters that we can introduce. Unfortunately, to my eyes, the syntax of that configuration file is a bit obscure.

And that is why it would be good to have a simple GUI option to enable CUDA processing :)

A simple "Use CUDA" checkbox in the option section of the GUI would do! FFmetrics could detect if CUDA is supported on the system and disable the checkbox if no CUDA GPU is available.

[fifonik]

If you'd like to use libvmaf_cuda instead of libvmaf you should create FFMetrics.conf with the following content:

{
    "Metrics": {
        "VMAF": {
            "Filter": "libvmaf_cuda=eof_action=endall:n_threads={{n_threads}}:log_fmt={{log_fmt}}:log_path='{{log_path}}':pool={{pool}}:phone_model={{phone_model}}:model_path='{{model_path}}':model=version={{model_version}}"
        }
    },
    "Metric": {
        "Template": "-hide_banner -nostdin -r {{main-framerate}} -hwaccel cuda -hwaccel_output_format cuda -i {{main-src}} -r {{ref-framerate}} -hwaccel cuda -hwaccel_output_format cuda -i {{ref-src}} -frames:v {{duration}} -lavfi {{lavfi}} -f null -"
    }
}

Assuming the libvmaf_cuda supports exactly the same parameters as libvmaf. I do not know if the -r should be before the -hwaccell* or after.

P.S. I'm unable to test it as I do not have any NVidia GPU.

[tormento]

Unfortunately that configuration disables VMAF.

[tormento]

It seems, anyway, that ffmpeg is already using some GPU even in PSNR. Just decoding, I suppose.

[fifonik]

disables VMAF.

What does it mean? These lines not used for feature detection so they cannot "disable VMAF" (I mean disable the VMAF checkbox so user will not be able to tick it to calculate the metric). Any useful info from log?

P.S. I mentioned that I do not have NVidia GPU so unable to test it.

[tormento]

I have created a new FFmetric.conf containing just the lines you posted.

Result:

VMAF is greyed out and not shown in FFMPEG supported metrics.

Log:

2024-03-19 11:43:01 ERROR: Cannot process configuration file D:\Eseguibili\Media\FFMetrics\FFMetrics.conf: After parsing a value an unexpected character was encountered: ". Path 'Metr4ic', line 20, position 1.
2024-03-19 11:45:08 WARNING: VMAF metric initialization failed

[fifonik]

The 'Cannot process configuration file' error is weird. This means your conf contains invalid json while mine (posted above) is correct. If I copy the json I posted and paste into a new FFMetrics.conf and then replace 'libvmav_cuda' with 'linvmaf', FFMetrics opened with no issues (I'm using the latest version of FFMetrics). If I paste it "as is" and keep libvmaf_cuda, I only have the VMAF metric initialization failed error message and disabled checkbox with one version of FFMpeg and all good with another version of FFMpeg. I was wrong in my above message, FFMetrics is actually testing VMAF and its models availability.

To check if the FFMpeg you use is actually does not have issues with using the libvmaf_cuda library, you can do the following:

1. Tick Options | Write FFMpeg comands into log and restart FFMetrics.
2. Open the log and copy the line: ffmpeg -hide_banner -nostdin -f lavfi -i color=c=black:s=320x240:d=1 -lavfi [0:v][0:v]libvmaf_cuda=eof_action=endall:n_threads=15:model=version=vmaf_v0.6.1 -f null -
3. Try to run in from command prompt. It should not give any errors.

Unfortunately, I do not know what version of FFMpeg required for using the libvmaf_cuda and if any additional dlls needed.

[fifonik]

Here is my output:

[tormento]

I have your very same error, both with Btbn and Gyan builds.

[fifonik]

This means FFMpeg you use do not support the libvmaf_cuda. You may need to find a special buld or some additional dlls if they required (google might help). Unfortunately, I cannot help you with that as I do not have required hardware.

[Ptilopsis01]

This is the CUDA variant of the libvmaf filter. It only accepts CUDA frames.

I've found this in ffmpeg document. Maybe it's necessary to convert the input through CUDA filter before using vmaf_cuda?

[fifonik]

If you check my suggestion about FFMetrics.conf you will see that I added the convertion you mentioned there as it is present in libvmaf_cuda's FFMpeg manual.

When someone find a way to calculate VMAF using cuda HW acceleration and supply link to ffmpeg.exe that can do this and command line -- I will be able to help with setting up FFMetrics.

P.S. I believe that if format is the issue, the error message would not be 'no such filter: libvmaf_cuda'

[Ptilopsis01]

After installing the library it can be enabled using: ./configure --enable-nonfree --enable-ffnvcodec --enable-libvmaf

It says that libvmaf_cuda needs the above configure options when building ffmpeg in the official document, but

--enable-nonfree allow use of nonfree code, the resulting libs and binaries will be unredistributable

So I'm afraid there won't be any release of ffmpeg with libvmaf_cuda on😂

[fifonik]

Well, then someone will need to compile FFMpeg with libvmaf_cuda support, play with command line and find a way to use the filter. After that I will be able to assist with setting up FFMetrics.conf to use it.