search

Netflix / vmaf

Perceptual video quality assessment based on multi-method fusion.
Other
4.65k stars 755 forks source link

libvmaf won't statically build into ffmpeg #788

Open Provissy opened 3 years ago

Provissy commented 3 years ago

Description

If build ffmpeg with libvmaf, libvmaf won't be compiled into ffmpeg staticlly. Instead, ffmpeg still needs libvmaf.so.1 as runtime dependency.

How to reproduce

Build ffmpeg statically in a container.

# build libvmaf
meson build --default-library static --buildtype release
ninja -C build
ninja -C build install

# build ffmpeg statically 
PKG_CONFIG_PATH="/usr/local/lib64/pkgconfig:/usr/local/lib/pkgconfig" ./configure \
  --pkg-config-flags="--static" \
  --extra-cflags="-I/usr/local/include" \
  --extra-ldflags="-L/usr/local/lib" \
  --extra-ldflags="-L/usr/local/lib64" \
  --extra-libs=-pthread \
  --extra-libs=-lm \
  --enable-gpl \
  --enable-libvmaf
make -j$(nproc)
make install

Now get the ffmpeg out from the container, run it, the result is ./ffmpeg: error while loading shared libraries: libvmaf.so.1: cannot open shared object file: No such file or directory

This problem occurs even if runnig ffmpeg inside the build environment, which means inside the container in this case.

Note

Might be related to https://github.com/Netflix/vmaf/issues/753#issuecomment-742002566 https://github.com/Netflix/vmaf/issues/557

1480c1 commented 3 years ago

Can you try patching vmaf with https://github.com/Netflix/vmaf/pull/768? you can wget https://github.com/Netflix/vmaf/pull/768.patch and using git apply with it

Provissy commented 3 years ago

Can you try patching vmaf with #768? you can wget https://github.com/Netflix/vmaf/pull/768.patch and using git apply with it

Hmm, after patched, ./configure of ffmpeg gives this

ERROR: libvmaf >= 1.5.2 not found using pkg-config

No error if not patched.

More info

[root@3863aa09c895 libvmaf]# meson build --default-library static --buildtype release -Denable_tests=false -Denable_docs=
false
The Meson build system
Version: 0.49.2
Source dir: /root/vmaf/libvmaf
Build dir: /root/vmaf/libvmaf/build
Build type: native build
Project name: libvmaf
Project version: 2.0.0
Appending CFLAGS from environment: '-march=native'
Native C compiler: /usr/bin/clang (clang 10.0.1 "clang version 10.0.1 (Red Hat 10.0.1-1.module+el8.3.0+7459+90c24896)")
Appending CXXFLAGS from environment: '-march=native'
Native C++ compiler: /usr/bin/clang++ (clang 10.0.1 "clang version 10.0.1 (Red Hat 10.0.1-1.module+el8.3.0+7459+90c24896)")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: NO
Configuring config.h using configuration
Dependency threads found: YES
Library m found: YES
Checking for function "strsep" : YES
Build targets in project: 7
Option default_library is: static [default: both]
Found ninja-1.8.2 at /usr/bin/ninja
Provissy commented 3 years ago

Tried your CI configuration of ffmpeg here https://github.com/AOMediaCodec/SVT-AV1/pull/1647/files

After adding --ld=g++ , ./configure now works.

if no --ld=g++ specified, configure fails no matter using clang or gcc as CC.

tail of ffbuild/config.log

/opt/rh/gcc-toolset-10/root/usr/bin/ld: svm.cpp:(.text._ZN14SVMModelParserI24SVMModelParserFileSourceE21parse_support_vectorsEv[_ZN14SVMModelParserI24SVMModelParserFileSourceE21parse_support_vectorsEv]+0x895): undefined reference to `std::__cxx11::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_istringstream()'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI7QMatrix[_ZTI7QMatrix]+0x0): undefined reference to `vtable for __cxxabiv1::__class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI6Kernel[_ZTI6Kernel]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI6Solver[_ZTI6Solver]+0x0): undefined reference to `vtable for __cxxabiv1::__class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI9Solver_NU[_ZTI9Solver_NU]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI5SVC_Q[_ZTI5SVC_Q]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI11ONE_CLASS_Q[_ZTI11ONE_CLASS_Q]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTI5SVR_Q[_ZTI5SVR_Q]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTV6Kernel[_ZTV6Kernel]+0x10): undefined reference to `__cxa_pure_virtual'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.ro._ZTV6Kernel[_ZTV6Kernel]+0x18): undefined reference to `__cxa_pure_virtual'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.local.DW.ref._ZTISt13runtime_error[DW.ref._ZTISt13runtime_error]+0x0): undefined reference to `typeinfo for std::runtime_error'
/opt/rh/gcc-toolset-10/root/usr/bin/ld: /usr/local/lib/libvmaf.a(svm.cpp.o):(.data.rel.local.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0'
collect2: error: ld returned 1 exit status
ERROR: libvmaf >= 1.5.2 not found using pkg-config
kocoten1992 commented 3 years ago

struggle for the last 2 days the same issue, have you solve it yet @Provissy ?

I got exact same issue, but even downgrade libvmaf down to 1.5.2, I running into old error again.

./ffmpeg: error while loading shared libraries: libvmaf.so.1: cannot open shared object file: No such file or directory
Provissy commented 3 years ago

struggle for the last 2 days the same issue, have you solve it yet @Provissy ?

I got exact same issue, but even downgrade libvmaf down to 1.5.2, I running into old error again.

./ffmpeg: error while loading shared libraries: libvmaf.so.1: cannot open shared object file: No such file or directory

Use the latest git master brach to build libvmaf, the patch has been merged several days ago. https://github.com/Netflix/vmaf/pull/768

You should append --ld=g++ or --ld=clang++ when configuring ffmpeg.

It seems that ffmpeg uses default ld to link libvmaf objects which contains C++ symbols.

You can refer to the steps here https://github.com/AOMediaCodec/SVT-AV1/pull/1647/files

These steps should success...

export PKG_CONFIG_PATH="/usr/local/lib64/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig" 

cd vmaf/libvmaf
meson build --default-library=static --buildtype=release -Denable_tests=false -Denable_docs=false -Dbuilt_in_models=true
ninja -vC build
ninja -vC build install

cd ffmpeg
./configure \
  --ld="g++" \
  --pkg-config-flags="--static" \
  --enable-static \
  --extra-cflags="-I/usr/local/include -I/usr/lib/include" \
  --extra-cxxflags="-I/usr/local/include -I/usr/lib/include" \
  --extra-ldflags="-L/usr/local/lib -L/usr/local/lib64 -L/usr/lib -L/usr/lib64" \
  --extra-libs=-pthread \
  --extra-libs=-lm \
 --enable-libvmaf
nanake commented 3 years ago

if that's the case. why the workflow itself successfully build without ld=g++?

1480c1 commented 3 years ago

That's because it's linking libvmaf shared, so gcc, the linker ffmpeg uses by default, doesn't need to resolve all symbols

nanake commented 3 years ago

That's because it's linking libvmaf shared, so gcc, the linker ffmpeg uses by default, doesn't need to resolve all symbols

ahhh, above and this comment shred me some light.

Yes, and the problem is that one cannot build a static library alone. The dynamic lib is ALWAYS compiled and installed.

That's why I don't see any error prior to commit c171b64. So, I actually do only need --default-library=both instead of --default-library=static.

Without --default-library=both, I cannot compile libaom with -DCONFIG_TUNE_VMAF=1. It triggers error

Building C object CMakeFiles/aom_dsp_encoder.dir/aom_dsp/vmaf.c.obj /aom/aom_dsp/vmaf.c:16:10: fatal error: libvmaf.h: No such file or directory 16 | #include | ^~~

even though libvmaf.h exists and I can successfully build into ffmpeg without error ERROR: libvmaf >= 1.5.2 not found using pkg-config.

iamsimakov commented 3 years ago

So, I actually do only need --default-library=both instead of --default-library=static. @nanake thank you for idea. now I tried tio build latest version ffmpeg 4.3.1 and Vmaf 2.1.1 and caught ERROR: libvmaf >= 1.3.9 not found using pkg-config so --default-library=both when building vmaf was needed if you build DYNAMIC ffmpeg binary

To build static ffmpeg with vmaf you need --ld="g++" in ./configure command ffmpeg. And for any OS - Centos/Ubuntu you should update compiler gcc, it should be > 4.8.5 version

rdp commented 2 years ago

My work arounds for these were to (well I changed it so it was building a .dll) delete the installed .dll.a file so that it would only "have installed" static, and to add -lstdc++ to the .pc file so that it would link right (not have the undefined reference to vtable for __cxxabiv1) ... :) I think the lstdc++ issue is still there, at least, in 2.3.0. Cheers!