[Request] Enable libvmaf_cuda filter

[HunterAP23]

VMAF added support for CUDA acceleration, followed by FFmpeg adding support for creating the libvmaf_cuda filter: https://github.com/FFmpeg/FFmpeg/commit/7f685d0f493bbfa44cf1b3b65f9347291e23872b

The option for building libvmaf with CUDA support is -Denable_cuda=true in the libvmaf's meson build chain. This requires CUDA and NVCC to compile. FFmpeg needs to be configured with these options to work: --enable-nonfree --enable-ffnvcodec --enable-libvmaf

Would it be possible to add the libvmaf_cuda filter in the full git builds?

[GyanD]

If --enable-nonfree is accurate, then a build with that filter cannot be legally distributed. It has to be self-compiled.

That may stem from nvcc. Can't it be compiled using llvm?

[HunterAP23]

I do believe this is due to needing nvcc for the build process, since that's what the meson/ninja configuration for libvmaf specifically look for, but I can't confirm it since libvmaf currently fails to compile. When setting up the environment, installing cuda, and using the ffmpeg-windows-build-helpers tool, libvmaf fails to compile as it seems to not be able to find CUDA when using the option --cross-file=meson-cross.mingw.txt:

The Meson build system
Version: 1.2.3
Source dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf
Build dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build
Build type: cross build
Project name: libvmaf
Project version: 2.3.1
C compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc (gcc 10.2.0 "x86_64-w64-mingw32-gcc (GCC) 10.2.0")
C linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc ld.bfd 2.36.1
C++ compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ (gcc 10.2.0 "x86_64-w64-mingw32-g++ (GCC) 10.2.0")
C++ linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ ld.bfd 2.36.1
C compiler for the build machine: cc (gcc 7.5.0 "cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the build machine: cc ld.bfd 2.30
C++ compiler for the build machine: c++ (gcc 7.5.0 "c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the build machine: c++ ld.bfd 2.30
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: YES (/usr/bin/xxd)
Run-time dependency CUDA found: NO

src/meson.build:162:22: ERROR: Dependency lookup for cuda with method 'system' failed: Couldn't find requested CUDA module 'cudart_static'

A full log can be found at /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build/meson-logs/meson-log.txt

I crudely modified the cross_compile_ffmpeg.sh file to grab the master branch of the VMAF library and to use the -Denable_cuda=true option in the meson setup, which is where the issue mentioned above occurs. I'm not sure how to set some variable for meson to find CUDA correctly in this state.

[GyanD]

Give me around a week to look into this.

[HunterAP23]

Providing some more information that I've come across. I fixed the above issue by copying the libcudart_static.a file that gets added to my system when installing Cuda 11.6 through the cuda-toolkit-11-6 package. For me this file is in both /usr/local/cuda-11.6/targets/x86_64-linux/lib and /usr/lib/x86_64-linux-gnu, and I copied this file to the ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32/lib directory.

After doing this, the cross_compile_ffmpeg.sh script to change what normally gets written to meson-cross.mingw.txt by changing the following section:

[binaries]
c = '${cross_prefix}gcc'
cpp = '${cross_prefix}g++'
ld = '${cross_prefix}ld'
ar = '${cross_prefix}ar'
strip = '${cross_prefix}strip'
pkgconfig = '${cross_prefix}pkg-config'
nm = '${cross_prefix}nm'
windres = '${cross_prefix}windres'

and adding cuda = '/usr/bin/nvcc' to the bottom. This does confirm that nvcc is required so this should make the build nonfree.

Afterwards attempting to run the compile script again, I'm met with a different error, this one from libvmaf itself:

[2/19] Linking target test/test_picture.exe
FAILED: test/test_picture.exe
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++  -o test/test_picture.exe test/test_picture.exe.p/test.c.obj test/test_picture.exe.p/test_picture.c.obj test/test_picture.exe.p/.._src_picture.c.obj test/test_picture.exe.p/.._src_mem.c.obj test/test_picture.exe.p/.._src_ref.c.obj test/test_picture.exe.p/.._src_thread_pool.c.obj -Wl,--allow-shlib-undefined -Wl,-O1 -pthread -Wl,--start-group -lcuda /home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libcudart_static.a -L/usr/local/cuda/lib/x64 -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status
[3/19] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
/usr/bin/nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

Not sure what the -lcuda option does specifically, but it seems to look for CUDA even though it was able to find it earlier.

I'm relaying this info to the both the ffmpeg-windows-build-helpers repo and to VMAF as well.

[GyanD]

So, I had a look inside vmaf's build system and it specifically needs nvcc. So that will then require nonfree flag in ffmpeg and thus makes this moot.

I can find an old issue where it is discussed to switch to the Nvidia headers that ffmpeg maintains, which would allow llvm for compilation, but it hasn't progressed since.