arthenica / ffmpeg-kit

FFmpeg Kit for applications. Supports Android, Flutter, iOS, Linux, macOS, React Native and tvOS. Supersedes MobileFFmpeg, flutter_ffmpeg and react-native-ffmpeg.
https://arthenica.github.io/ffmpeg-kit
GNU Lesser General Public License v3.0
4.5k stars 602 forks source link

Multiple Overlay Filters Causing Memory Error - Flutter (Only for iOS) #375

Closed ibrahimdevs closed 2 years ago

ibrahimdevs commented 2 years ago

Description I need to put overlay images to a video. It is working on Android without problem. But on iOS platform, if I try 23-24 overlay images, it is working correctly. If I try it with 30+ images, it gives memory allocation error.

Most likely related with: #121 and #329

FFmpeg Command: -i video.mov -i image1.png -i image1.png -i ... image43.png
-filter_complex "[0][1]overlay=x=0:y=0:enable='between(t,10.9,16.4)'[out];[out][2]overlay=x=0:y=0:enable='between(t,16.8,20.0)'[out];[out][3]overlay=x=0:y=0:enable='between(t,20.0,22.4)'[out];[out][4]overlay=x=0:y=0:enable='between(t,22.4,26.5)'[out];[out][5]overlay=x=0:y=0:enable='between(t,26.5,29.8)'[out];[out][6]overlay=x=0:y=0:enable='between(t,30.0,33.7)'[out];[out][7]overlay=x=0:y=0:enable='between(t,33.7,38.4)'[out];[out][8]overlay=x=0:y=0:enable='between(t,38.6,41.5)'[out];[out][9]overlay=x=0:y=0:enable='between(t,41.5,44.2)'[out];[out][10]overlay=x=0:y=0:enable='between(t,44.3,46.5)'[out];[out][11]overlay=x=0:y=0:enable='between(t,47.2,48.4)'[out];[out][12]overlay=x=0:y=0:enable='between(t,49.9,52.6)'[out];[out][13]overlay=x=0:y=0:enable='between(t,52.6,54.2)'[out];[out][14]overlay=x=0:y=0:enable='between(t,54.2,57.0)'[out];[out][15]overlay=x=0:y=0:enable='between(t,57.0,57.6)'[out];[out][16]overlay=x=0:y=0:enable='between(t,63.0,65.6)'[out];[out][17]overlay=x=0:y=0:enable='between(t,66.5,71.5)'[out];[out][18]overlay=x=0:y=0:enable='between(t,72.2,73.7)'[out];[out][19]overlay=x=0:y=0:enable='between(t,74.4,75.7)'[out];[out][20]overlay=x=0:y=0:enable='between(t,77.9,78.7)'[out];[out][21]overlay=x=0:y=0:enable='between(t,83.5,85.4)'[out];[out][22]overlay=x=0:y=0:enable='between(t,87.3,91.0)'[out];[out][23]overlay=x=0:y=0:enable='between(t,91.0,96.1)'[out];[out][24]overlay=x=0:y=0:enable='between(t,96.1,100.8)'[out];[out][25]overlay=x=0:y=0:enable='between(t,100.8,103.5)'[out];[out][26]overlay=x=0:y=0:enable='between(t,103.5,106.9)'[out];[out][27]overlay=x=0:y=0:enable='between(t,106.9,109.1)'[out];[out][28]overlay=x=0:y=0:enable='between(t,109.2,110.6)'[out];[out][29]overlay=x=0:y=0:enable='between(t,110.6,113.7)'[out];[out][30]overlay=x=0:y=0:enable='between(t,114.8,117.6)'[out];[out][31]overlay=x=0:y=0:enable='between(t,117.6,119.8)'[out];[out][32]overlay=x=0:y=0:enable='between(t,120.5,122.8)'[out];[out][33]overlay=x=0:y=0:enable='between(t,123.5,125.0)'[out];[out][34]overlay=x=0:y=0:enable='between(t,128.3,129.1)'[out];[out][35]overlay=x=0:y=0:enable='between(t,129.7,130.5)'[out];[out][36]overlay=x=0:y=0:enable='between(t,131.4,136.3)'[out];[out][37]overlay=x=0:y=0:enable='between(t,137.1,138.6)'[out];[out][38]overlay=x=0:y=0:enable='between(t,139.3,139.9)'[out];[out][39]overlay=x=0:y=0:enable='between(t,143.2,143.6)'[out];[out][40]overlay=x=0:y=0:enable='between(t,148.8,152.2)'[out];[out][41]overlay=x=0:y=0:enable='between(t,152.2,155.3)'[out];[out][42]overlay=x=0:y=0:enable='between(t,155.3,158.2)'[out];[out][43]overlay=x=0:y=0:enable='between(t,158.2,159.2)'[out]" -map [out] -map 0:a? -vsync vfr -vcodec libx264 -x264-params crf=23 -preset superfast -acodec aac output.mp4

Expected behavior Exporting a video with different time based image overlays.

Current behavior Giving memory error.

To Reproduce You can try it with 30+ images which sizes per 50kb+ over Full HD video.

Logs ffmpeg version v4.5-dev-3393-g30322ebe3c Copyright (c) 2000-2021 the FFmpeg developers built with Apple clang version 13.0.0 (clang-1300.0.29.30) configuration: --cross-prefix=arm64-ios-darwin- --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk --prefix=/Users/taner/Projects/ffmpeg-kit/prebuilt/apple-ios-arm64/ffmpeg --pkg-config=/opt/homebrew/bin/pkg-config --enable-version3 --arch=aarch64 --cpu=armv8 --target-os=darwin --enable-neon --enable-asm --ar=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar --cc=clang --cxx=clang++ --as='/Users/taner/Projects/ffmpeg-kit/.tmp/gas-preprocessor.pl -arch aarch64 -- clang -arch arm64 -target arm64-apple-ios12.1 -march=armv8-a+crc+crypto -mcpu=generic -DFFMPEG_KIT_ARM64 -Wno-unused-function -Wno-deprecated-declarations -fstrict-aliasing -fembed-bitcode -DIOS -DFFMPEG_KIT_BUILD_DATE=20220114 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk -Oz -miphoneos-version-min=12.1 -I/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk/usr/include' --ranlib=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib --strip=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip --nm=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm --extra-ldflags='-miphoneos-version-min=12.1' --disable-autodetect --enable-cross-compile --enable-pic --enable-inline-asm --enable-optimizations --enable-swscale --enable-shared --disable-static --install-name-dir='@rpath' --enable-pthreads --disable-v4l2-m2m --disable-outdev=v4l2 --disable-outdev=fbdev --disable-indev=v4l2 --disable-indev=fbdev --enable-small --disable-xmm-clobber-test --disable-debug --disable-neon-clobber-test --disable-programs --disable-postproc --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-sndio --disable-schannel --disable-securetransport --disable-xlib --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --disable-alsa --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gmp --enable-gnutls --enable-libmp3lame --enable-libass --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libopencore-amrnb --enable-libshine --enable-libspeex --enable-libdav1d --enable-libkvazaar --enable-libx264 --enable-libxvid --enable-libx265 --enable-libvidstab --enable-libilbc --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libtwolame --disable-sdl2 --enable-libvo-amrwbenc --enable-libzimg --disable-openssl --enable-zlib --enable-audiotoolbox --disable-outdev=audiotoolbox --enable-bzlib --enable-videotoolbox --enable-avfoundation --enable-iconv --disable-coreimage --disable-appkit --disable-opencl --disable-opengl --enable-gpl libavutil 57. 13.100 / 57. 13.100 libavcodec 59. 15.102 / 59. 15.102 libavformat 59. 10.100 / 59. 10.100 libavdevice 59. 1.100 / 59. 1.100 libavfilter 8. 21.100 / 8. 21.100 libswscale 6. 1.102 / 6. 1.102 libswresample 4. 0.100 / 4. 0.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mov': Metadata: major_brand : qt
minor_version : 0 compatible_brands: qt
creation_time : 2022-03-19T16:24:25.000000Z encoder : Lavf58.20.100 Duration: 00:02:47.77, start: 0.000000, bitrate: 12431 kb/s Stream #0:00x1: Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 248 kb/s (default) Metadata: creation_time : 2022-03-19T16:24:25.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:10x2: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 1080x1920 [SAR 1:1 DAR 9:16], 12174 kb/s, 29.99 fps, 30 tbr, 90k tbn (default) Metadata: creation_time : 2022-03-19T16:24:25.000000Z handler_name : Core Media Video vendor_id : FFMP Input #1, png_pipe, from 'image1.png': Duration: N/A, bitrate: N/A Stream #1:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn Input #2, png_pipe, from 'image2.png': Duration: N/A, bitrate: N/A Stream #2:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn . . Input #43, png_pipe, from 'image43.png': Duration: N/A, bitrate: N/A Stream #43:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn [aac @ 0x1047f4480] The "sub_text_format" option is deprecated: Deprecated, does nothing [h264 @ 0x15f2a70f0] The "sub_text_format" option is deprecated: Deprecated, does nothing [png @ 0x15f47f710] The "sub_text_format" option is deprecated: Deprecated, does nothing [png @ 0x1059da740] The "sub_text_format" option is deprecated: Deprecated, does nothing . . [png @ 0x15f2a8780] The "sub_text_format" option is deprecated: Deprecated, does nothing Stream mapping: Stream #0:1 (h264) -> scale (graph 0) Stream #1:0 (png) -> overlay:overlay (graph 0) Stream #2:0 (png) -> overlay:overlay (graph 0) Stream #3:0 (png) -> overlay:overlay (graph 0) Stream #4:0 (png) -> overlay:overlay (graph 0) Stream #5:0 (png) -> overlay:overlay (graph 0) Stream #6:0 (png) -> overlay:overlay (graph 0) Stream #7:0 (png) -> overlay:overlay (graph 0) Stream #8:0 (png) -> overlay:overlay (graph 0) Stream #9:0 (png) -> overlay:overlay (graph 0) Stream #10:0 (png) -> overlay:overlay (graph 0) Stream #11:0 (png) -> overlay:overlay (graph 0) Stream #12:0 (png) -> overlay:overlay (graph 0) Stream #13:0 (png) -> overlay:overlay (graph 0) Stream #14:0 (png) -> overlay:overlay (graph 0) Stream #15:0 (png) -> overlay:overlay (graph 0) Stream #16:0 (png) -> overlay:overlay (graph 0) Stream #17:0 (png) -> overlay:overlay (graph 0) Stream #18:0 (png) -> overlay:overlay (graph 0) Stream #19:0 (png) -> overlay:overlay (graph 0) Stream #20:0 (png) -> overlay:overlay (graph 0) Stream #21:0 (png) -> overlay:overlay (graph 0) Stream #22:0 (png) -> overlay:overlay (graph 0) Stream #23:0 (png) -> overlay:overlay (graph 0) Stream #24:0 (png) -> overlay:overlay (graph 0) Stream #25:0 (png) -> overlay:overlay (graph 0) Stream #26:0 (png) -> overlay:overlay (graph 0) Stream #27:0 (png) -> overlay:overlay (graph 0) Stream #28:0 (png) -> overlay:overlay (graph 0) Stream #29:0 (png) -> overlay:overlay (graph 0) Stream #30:0 (png) -> overlay:overlay (graph 0) Stream #31:0 (png) -> overlay:overlay (graph 0) Stream #32:0 (png) -> overlay:overlay (graph 0) Stream #33:0 (png) -> overlay:overlay (graph 0) Stream #34:0 (png) -> overlay:overlay (graph 0) Stream #35:0 (png) -> overlay:overlay (graph 0) Stream #36:0 (png) -> overlay:overlay (graph 0) Stream #37:0 (png) -> overlay:overlay (graph 0) Stream #38:0 (png) -> overlay:overlay (graph 0) Stream #39:0 (png) -> overlay:overlay (graph 0) Stream #40:0 (png) -> overlay:overlay (graph 0) Stream #41:0 (png) -> overlay:overlay (graph 0) Stream #42:0 (png) -> overlay:overlay (graph 0) Stream #43:0 (png) -> overlay:overlay (graph 0) overlay (graph 0) -> Stream #0:0 (libx264) Stream #0:0 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help Error while filtering: Cannot allocate memory Failed to inject frame into filter network: Cannot allocate memory Error while processing the decoded data for stream #43:0 [aac @ 0x15f23b480] Qavg: 12511.496 [aac @ 0x15f23b480] 2 frames left in the queue on closing Conversion failed!

Environment Flutter

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/ibrahim/Library/Android/sdk • Platform android-32, build-tools 31.0.0 • ANDROID_HOME = /Users/ibrahim/Library/Android/sdk • Java binary at: /Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822) • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.3) • Xcode at /Applications/Xcode.app/Contents/Developer • CocoaPods version 1.11.2

[✓] Android Studio (version 2021.1) • Android Studio at /Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] Android Studio (version 2021.1) • Android Studio at /Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] VS Code (version 1.65.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension can be installed from: 🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (1 available) • device X (mobile) • 2375e3716bb5cc7b915c93fa8e122f93f47e71d3 • ios • iOS 15.3.1 19D52

[✓] HTTP Host Availability • All required HTTP hosts are available

• No issues found!`

ibrahimdevs commented 2 years ago

I solved this issue with reducing the overlay filters. You can learn about more from my stackoverflow question. https://stackoverflow.com/questions/71540219/ffmpegkit-multiple-overlay-filters-causing-memory-error-flutter-only-for-ios/71543080#71543080