search

tanersener / react-native-ffmpeg

FFmpeg for react-native. Not maintained anymore. Superseded by FFmpegKit.
GNU General Public License v3.0
401 stars 52 forks source link

libx264 crashes the app each time #152

Closed jeb218 closed 4 years ago

jeb218 commented 4 years ago

Description On full-gpl if I am using mpeg4 everything works great, meaning the video is processed and ffmpeg completes, no problems. I think mpeg4 lacks in quality, and takes longer to process, but that is beside the point.

Also on full-gpl, using libx264, the processing always crashes after about 55 frames - the app closes, and no errors are output to the log so I can't see what is causing it. If possible, I'd like to get libx264 to work because the processing fps is about 3-4 times faster than mpeg4.

Do you have any luck using libx264, or have any ideas on what might be causing the process to fail?

Thanks again!

Expected behavior encoding to complete without crashing, like when using mpeg4

Current behavior encoding crashes after about 55 frames on libx264

Screenshots If applicable, add screenshots to help explain your problem.

Logs LOG file:///data/user/0/.../cache/Camera/63c7013f-7c48-47ea-a048-05a285505d25.mp4 LOG ["-i", "file:///data/user/0/.../cache/Camera/63c7013f-7c48-47ea-a048-05a285505d25.mp4", "-vcodec", "libx264", "-vf", "drawtext=enable='between(t,3,5)':fontfile='/system/fonts/DancingScript-Bold.ttf':fontsize=50:fontcolor=#cc0000:text='Hello CV World':x=(w-text_w)/2:y=(h-text_h)/2", "-c:a", "copy", "-movflags", "+faststart", "/storage/emulated/0/DCIM/Camera/myvid_postprocess_389.mp4"] LOG ffmpeg version git-2020-01-25-fd11dd500 LOG Copyright (c) 2000-2020 the FFmpeg developers LOG

LOG built with Android (5220042 based on r346389c) clang version 8.0.7 (https://android.googlesource.com/toolchain/clang b55f2d4ebfd35bf643d27dbca1bb228957008617) (https://android.googlesource.com/toolchain/llvm 3c393fe7a7e13b0fba4ac75a01aa683d7a5b11cd) (based on LLVM 8.0.7svn)

LOG configuration: --cross-prefix=aarch64-linux-android- --sysroot=/files/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/sysroot --prefix=/home/taner/Projects/mobile-ffmpeg/prebuilt/android-arm64/ffmpeg --pkg-config=/usr/bin/pkg-config --enable-version3 --arch=aarch64 --cpu=armv8-a --cc=aarch64-linux-android24-clang --cxx=aarch64-linux-android24-clang++ --target-os=android --enable-neon --enable-asm --enable-inline-asm --enable-cross-compile --enable-pic --enable-jni --enable-optimizations --enable-swscale --enable-shared --disable-v4l2-m2m --disable-outdev=v4l2 --disable-outdev=fbdev --disable-indev=v4l2 --disable-indev=fbdev --enable-small --disable-openssl --disable-xmm-clobber-test --disable-debug --enable-lto --disable-neon-clobber-test --disable-programs --disable-postproc --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-static --disable-sndio --disable-schannel --disable-securetransport --disable-xlib --disable-cuda --disable-cuvid --disa LOG libavutil 56. 38.100 / 56. 38.100

LOG libavcodec 58. 65.102 / 58. 65.102

LOG libavformat 58. 35.101 / 58. 35.101

LOG libavdevice 58. 9.103 / 58. 9.103

LOG libavfilter 7. 70.101 / 7. 70.101

LOG libswscale 5. 6.100 / 5. 6.100

LOG libswresample 3. 6.100 / 3. 6.100

LOG Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:///data/user/0/.../cache/Camera/63c7013f-7c48-47ea-a048-05a285505d25.mp4':

LOG Metadata:

LOG major_brand : LOG mp42 LOG

LOG minor_version : LOG 0 LOG

LOG compatible_brands: LOG isommp42 LOG

LOG creation_time : LOG 2020-04-16T22:50:04.000000Z LOG

LOG com.android.version: LOG 10 LOG

LOG Duration: LOG 00:00:14.76 LOG , start: LOG 0.000000 LOG , bitrate: LOG 41228 kb/s LOG

LOG Stream #0:0 LOG (eng) LOG : Video: h264 (avc1 / 0x31637661), yuvj420p(pc, smpte170m/bt470bg/smpte170m), 3840x2160, 42208 kb/s LOG , SAR 1:1 DAR 16:9 LOG , LOG 29.75 fps, LOG 29.92 tbr, LOG 90k tbn, LOG 180k tbc LOG (default) LOG

LOG Metadata:

LOG rotate : LOG 90 LOG

LOG creation_time : LOG 2020-04-16T22:50:04.000000Z LOG

LOG handler_name : LOG VideoHandle LOG

LOG Side data:

LOG
LOG displaymatrix: rotation of -90.00 degrees LOG

LOG Stream #0:1 LOG (eng) LOG : Audio: aac (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s LOG (default) LOG

LOG Metadata:

LOG creation_time : LOG 2020-04-16T22:50:04.000000Z LOG

LOG handler_name : LOG SoundHandle LOG

LOG Stream mapping:

LOG Stream #0:0 -> #0:0 LOG (h264 (native) -> h264 (libx264)) LOG

LOG Stream #0:1 -> #0:1 LOG (copy) LOG

LOG Press [q] to stop, [?] for help

LOG [graph 0 input from stream 0:0 @ 0x744a8ae500] sws_param option is deprecated and ignored

LOG [libx264 @ 0x744aa32c00] using SAR=1/1

LOG [libx264 @ 0x744aa32c00] using cpu capabilities: ARMv8 NEON

LOG [libx264 @ 0x744aa32c00] profile High, level 5.1, 4:2:0, 8-bit

LOG [libx264 @ 0x744aa32c00] 264 - core 159 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

LOG Output #0, mp4, to '/storage/emulated/0/DCIM/Camera/myvid_postprocess_389.mp4':

LOG Metadata:

LOG major_brand : LOG mp42 LOG

LOG minor_version : LOG 0 LOG

LOG compatible_brands: LOG isommp42 LOG

LOG com.android.version: LOG 10 LOG

LOG encoder : LOG Lavf58.35.101 LOG

LOG Stream #0:0 LOG (eng) LOG : Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 2160x3840 [SAR 1:1 DAR 9:16], q=-1--1 LOG , LOG 29.92 fps, LOG 11488 tbn, LOG 29.92 tbc LOG (default) LOG

LOG Metadata:

LOG encoder : LOG Lavc58.65.102 libx264 LOG

LOG creation_time : LOG 2020-04-16T22:50:04.000000Z LOG

LOG handler_name : LOG VideoHandle LOG

LOG Side data:

LOG
LOG cpb: LOG bitrate max/min/avg: 0/0/0 buffer size: 0 LOG vbv_delay: N/A LOG

LOG
LOG displaymatrix: rotation of -0.00 degrees LOG

LOG Stream #0:1 LOG (eng) LOG : Audio: aac (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 96 kb/s LOG (default) LOG

LOG Metadata:

LOG creation_time : LOG 2020-04-16T22:50:04.000000Z LOG

LOG handler_name : LOG SoundHandle LOG

LOG frame= 4 fps=0.0 q=0.0 size= 0kB time=00:00:01.02 bitrate= 0.4kbits/s speed=1.74x
LOG frame= 8 fps=6.5 q=0.0 size= 0kB time=00:00:01.02 bitrate= 0.4kbits/s speed=0.826x
LOG frame= 12 fps=6.4 q=0.0 size= 0kB time=00:00:01.02 bitrate= 0.4kbits/s speed=0.549x
LOG frame= 16 fps=6.7 q=0.0 size= 0kB time=00:00:01.02 bitrate= 0.4kbits/s speed=0.428x
LOG frame= 22 fps=7.3 q=0.0 size= 0kB time=00:00:01.02 bitrate= 0.4kbits/s dup=1 drop=0 speed=0.341x
LOG frame= 27 fps=7.6 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.574x
LOG frame= 34 fps=8.2 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.492x
LOG frame= 40 fps=8.4 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.427x
LOG frame= 43 fps=8.0 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.377x
LOG frame= 45 fps=7.5 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.336x
LOG frame= 47 fps=3.1 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.136x
LOG frame= 49 fps=3.1 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.127x
LOG frame= 50 fps=3.0 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.121x
LOG frame= 52 fps=2.7 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.104x
LOG frame= 54 fps=2.6 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.0978x
LOG frame= 55 fps=2.6 q=0.0 size= 0kB time=00:00:02.02 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.0955x
LOG frame= 55 fps=2.4 q=0.0 size= 0kB time=00:00:02.04 bitrate= 0.2kbits/s dup=1 drop=0 speed=0.0885x

Environment OS: Linux 5.3 Ubuntu 19.10 (Eoan Ermine) Shell: 5.0.3 - /bin/bash Binaries: Node: 13.9.0 - ~/.nvm/versions/node/v13.9.0/bin/node Yarn: 1.22.4 - /usr/bin/yarn npm: 6.14.4 - ~/.nvm/versions/node/v13.9.0/bin/npm npmPackages: expo: ^37.0.7 => 37.0.7 react: ~16.9.0 => 16.9.0 react-native: ~0.61.5 => 0.61.5 npmGlobalPackages: expo-cli: 3.18.2

Other Note - even though expo is listed above, I am not using a managed workflow, i am using barebones react native with expo packages (I couldn't get ffmpeg to work on expo managed)

Also, I am using ffmpeg to process videos recorded on the fly by my phone's camera, so it is a different video each time I run it

tanersener commented 4 years ago

Can you please share one of the video files, so I can test it locally?

Also, can you try testing testing the default test-application from https://github.com/tanersener/react-native-ffmpeg-test repository. It has a video tab where you can create a slideshow with libx264 codec. It may help us to understand whether this issues is specific to your command & video.

jeb218 commented 4 years ago

Here is a dropbox link to one of them:

https://www.dropbox.com/s/ze25xebt6nv07k2/myvid_preprocess_296.mp4.zip?dl=0

I was able to use the test repo to create the slideshow and process it using the same commands.

jeb218 commented 4 years ago

Some other insight i found in my adb logcat *:E dump:

lowmemorykiller: Kill 'com.company.app' (19946), uid 10377, oom_adj 0 to free 1345876kB

tanersener commented 4 years ago

Is this com.company.app app your application?

jeb218 commented 4 years ago

yes it is. i changed what it is showing on github, but in the log it shows my app's package name there

tanersener commented 4 years ago

Do you know what lowmemorykiller is and what that message says?

jeb218 commented 4 years ago

it is an android system process https://source.android.com/devices/tech/perf/lmkd

So i'm assuming the libx264 codec is using a lot more RAM than the mpeg4 one does, and the device kills the process to free up memory

tanersener commented 4 years ago

Yeah, it looks like it is a memory issue. Application is not crashing, lowmemorykiller is killing it. You need to try to decrease the memory usage for your app.

jeb218 commented 4 years ago

I believe it is the the libx264 encoder that is using the majority of the ram. mpeg4 never runs into that issue

tanersener commented 4 years ago

I think you know the difference between mpeg4 and x264. So, it shouldn't be a surprise to see libx264 using more memory.

tanersener commented 4 years ago

By the way, I managed to successfully run your command with the video file you shared. So, I can confirm that the issue is not about a bug in react-native-ffmpeg. In terms of memory usage, you need to play with ffmpeg and x264 settings to decrease the memory usage. There does not exist a magic option/flag to decrease memory usage for you.

[22:44:56] I | ReactNativeJS ▶︎ Statistics; frame: 812.0, fps: 1.7, quality: 29.0, size: 74840649, time: 27520
[22:44:56] I | ReactNativeJS ▶︎ frame=  812 fps=1.7 q=-1.0 Lsize=   73087kB time=00:00:27.52 bitrate=21756.0kbits/s dup=7 drop=0 speed=0.0571x
[22:44:56] I | ReactNativeJS ▶︎ video:72734kB audio:323kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead:
[22:44:56] I | ReactNativeJS ▶︎ 0.041172%
jeb218 commented 4 years ago

interesting... the same command i used on the same file? Not sure why it won't process for me