BtbN / FFmpeg-Builds

MIT License
6.97k stars 956 forks source link

Which version to run on glibc 2.26 #365

Closed AdrKacz closed 2 months ago

AdrKacz commented 2 months ago

I try to run FFmpegon AWS Lambda which run on AL2. AL2 used glibc 2.26.

Unfortunately, the latest version for download requires glibc 2.28.

How could I find a version running under glibc 2.26.

Here's the output of ldd --versionin AL2

ldd (GNU libc) 2.26
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

Thank you for your help and your great work!

BtbN commented 2 months ago

The minimum glibc version is 2.28 through the board now, since more and more dependencies and finally ffmpeg itself started to heavily depend on C11 features, which are not fully supported until 2.28. So unfortunately support for systems older than that is not provided anymore, no matter which version you download.

AdrKacz commented 2 months ago

Thank you. How would you do to run FFmpeg on AL2 with the configuration enable-libfontconfig enable?

I need to use drawtext with a custom font (https://ffmpeg.org/ffmpeg-filters.html#drawtext-1)

I am open to alternatives. Thank you.

BtbN commented 2 months ago

I don't know what AL2 is, but glibc 2.28 came out in 2018. If you still are running a system which does not have it, it might be dire time to update it. The drawtext filter needs at least fontconfig and harfbuzz, so if you want to build an ffmpeg for that specific old distro, you'll have to install those packages via its package manager, and hope a recent ffmpeg still builds with the old toolchain and libraries.

AdrKacz commented 2 months ago

AL2 is Amazon Linux 2, a quite popular choice I'd say when your infrastructure is on Amazon Web Services.

When running on AWS Lambda with Python 3.11, it boots on AL2.

Could you please let me know how to build from scratch with the required packages? I tried to follow this guide but without success: https://trac.ffmpeg.org/wiki/CompilationGuide/Centos

If not with FFmpeg, how would you add overlay text with minimal dependencies installation?

Thank you for your help.

BtbN commented 2 months ago

You would need to build it on the distribution itself. Install the required dependencies and the compiler toolchain via the package manager, and then build ffmpeg manually. glibc 2.26 is really dang old, and Amazon using it in what appears to be the latest version is a huge issue. I'd look into alternatives to that distribution instead.

AdrKacz commented 2 months ago

Thank you for your help ☺️ I think I can get running on Amazon Linux 2023 which uses glibc 2.34 if I use Python 3.12 runtime. I need to check if it won't break any other dependencies though.

Thank you πŸ‘ (and good to know that 2.26 is very old, I didn't know)

BtbN commented 2 months ago

According to a quick google, 2.26 is from early 2017. You really don't want to be using a system that old anymore. It's also not a good sign for the general maintenance of that OS.

AdrKacz commented 2 months ago

Indeed πŸ‘

Quick follow up question, when downloading and packaging from your releases FFmpeg, ffmpeg and ffprobe and both a 45Mb executable.

The executable from John Van Sickle (https://johnvansickle.com) are 28Mb each. What explain this size difference (quite important)?

How could I do to release the size of your releases? The combine size of ffmpeg and ffprobe (90Mb) is too big to deploy on AWS Lambda (*was fine though with the 28Mb packages).

For example, is there a way I can pick only the configuration I need and remove unused files to reduce the total size?

Another solution won't be to get rid of the ffprobe export. I need ffprobe to read the bit_rate from my audio files. Do you know how could I do it without ffprobe? (so I can remove it from the AWS build and get within the size limit).

Thank you ☺️

BtbN commented 2 months ago

It's just the number of included dependencies. If you build yourself, you can freely choose what you include and what not.

abrar71 commented 2 months ago

@AdrKacz Also you can use the shared version of the build to reduce the total size by half, instead of using static version.

Andarwinux commented 2 months ago

You could also consider using patchelf hack to make a FFmpeg with bundled portable glibc, but I'm guessing hwaccel might not work.

AdrKacz commented 2 months ago

@abrar71 what are the differences between the shared and static version (beside the size)? And why is it named 'shared'.

AdrKacz commented 2 months ago

@BtbN do you have a guide to build it myself please? I tried to follow official instructions (https://trac.ffmpeg.org/wiki/CompilationGuide/Centos) but without success..

AdrKacz commented 2 months ago

It seems the latest version doesn't have libfontconfig enabled. Below are the configuration from https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl-shared.tar.xz.

You can see there is --enable-fontconfig but not --enable-libfontconfig. Do you know how can I enable it (see official documentation of drawtext that require --enable-libfontconfig: https://ffmpeg.org/ffmpeg-filters.html#drawtext-1)

To enable default font fallback and the font option you need to configure FFmpeg with --enable-libfontconfig.

configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-libs='-ldl -lgomp' --extra-ldflags=-pthread --extra-ldexeflags='-pie -Wl,-rpath=\$$ORIGIN -Wl,-rpath=\$$ORIGIN/../lib' --cc=x86_64-ffbuild-linux-gnu-gcc --cxx=x86_64-ffbuild-linux-gnu-g++ --ar=x86_64-ffbuild-linux-gnu-gcc-ar --ranlib=x86_64-ffbuild-linux-gnu-gcc-ranlib --nm=x86_64-ffbuild-linux-gnu-gcc-nm --extra-version=20240421
Andarwinux commented 2 months ago

@abrar71 what are the differences between the shared and static version (beside the size)? And why is it named 'shared'.

ffmpeg actually consists of several libav libraries, and ffmpeg, ffplay, ffprobe are part of fftools, the static build here means that fftools are statically linked to libav, and the dynamic build is dynamically linked to libav. ffmpeg dependencies are always statically linked, while platform standard libraries are always dynamically linked to prevent ABI problems. If you need to call libav or use more than one program in fftools, choose the dynamic build, otherwise choose the static build for best performance, convenience and smaller size.

BtbN commented 2 months ago

There is zero performance difference between the builds. The shared variant exists if you need the libraries, for use in other programs. If you want to run the CLI tools, I'd recommend to always stick to the fully static ones, to prevent library confusing/dll hell.

AdrKacz commented 2 months ago

Thank you @BtbN and @Andarwinux for the context πŸ‘ I'll stick with the static ones then.

Due to the large size of the static build for ffmpeg, I cannot package both ffmpeg and ffprobe in my AWS Lambda. However, I use ffprobe to read the bit_rate of an .mp3 so I can convert it to .m4a with the same rate (if the rates are different, I have problems down the road).

That's the only reason I need ffprobe. How would you do to achieve this result without importing the 45Mb of ffprobe? Can I achieve this with ffmpeg only, or with other native tools?

Thank you very much for your help, Sorry for the repeated questions, I'm blocked and you all know way more than I do regarding ffmpeg and related areas πŸ‘

AdrKacz commented 2 months ago

I found some light library: mutagen. However it gives slightly different results from ffprobe. For example, on a given .mp3 file -Github doesn't support mp3 attachment-, ffprobe returns 48072 and mutagen returns 48000. Do you know why those differences can occur? (I'm not very familiar with the concept of bitrate anyways, I recently learnt it could be variable which I still struggle to comprehend..)

I know we're getting side way, feel free to redirect me or close the issue if you prefer πŸ‘

abrar71 commented 2 months ago

@AdrKacz You can also build a smaller, stripped-down version of FFmpeg/FFprobe by compiling it yourself or hiring me or someone else to custom-build it for you. Which can easily bring the total size under 10MB

BtbN commented 2 months ago

In any case, I'm gonna close this issue, since it's beyond what an issue here is about.