Kickflip / kickflip-android-sdk

Kickflip Android SDK - Live Video Streaming to the Cloud
https://kickflip.io
Apache License 2.0
660 stars 212 forks source link

FFmpegWrapper error during long registration #35

Closed Arlem closed 8 years ago

Arlem commented 9 years ago

Hi, I am using sdk in my project and I am experiencing an error. I initially thought it was something related to the modifications I made to the sdk, so i tested your example app, but the problem still persists. The problem is that the index*.ts file stops being written and is not uploaded until the stop recording button is pressed. To reproduce the bug leave the device recording for at least 40 minutes. This is the resulting logcat (from Sony SGP611 on 5.0.2, I have another Logcat from Galaxy Camera 2 EK-GC200 with 4.3)

06-17 15:31:58.632  17126-17208/io.kickflip.sample E/FFmpegWrapper﹕ av_interleaved_write_frame video: 0 pkt: 64440 size: 565 error: Invalid argument
06-17 15:31:58.655  17126-17208/io.kickflip.sample E/FFmpegWrapper﹕ av_interleaved_write_frame video: 1 pkt: 64441 size: 25775 error: Invalid argument
06-17 15:31:58.668  17126-17208/io.kickflip.sample E/FFmpegWrapper﹕ av_interleaved_write_frame video: 0 pkt: 64441 size: 564 error: Invalid argument
06-17 15:31:58.691  17126-17208/io.kickflip.sample E/FFmpegWrapper﹕ av_interleaved_write_frame video: 1 pkt: 64442 size: 9889 error: Invalid argument
06-17 15:31:58.693  17126-17126/io.kickflip.sample I/MicrophoneEncoder﹕ stopRecording
06-17 15:31:58.693  17126-17126/io.kickflip.sample I/CameraEncoder﹕ stopRecording
06-17 15:31:58.705  17126-17223/io.kickflip.sample I/MicrophoneEncoder﹕ Exiting audio encode loop. Draining Audio Encoder
06-17 15:31:58.725  17126-17208/io.kickflip.sample E/FFmpegWrapper﹕ av_interleaved_write_frame video: 1 pkt: 64443 size: 9106 error: Invalid argument
06-17 15:31:58.737  17126-17217/io.kickflip.sample I/CameraEncoder﹕ Sending last video frame. Draining encoder
06-17 15:31:58.753  17126-17208/io.kickflip.sample I/FFmpegMuxer﹕ Forcing Shutdown
06-17 15:31:58.753  17126-17208/io.kickflip.sample I/FFmpegWrapper﹕ finalizeAVFormatContext
06-17 15:31:58.753  17126-17228/io.kickflip.sample D/HlsFileObserver﹕ Event 8 at index214.ts ext ts
06-17 15:31:58.754  17126-17228/io.kickflip.sample I/S3Manager﹕ Queueing upload ies_test/jo2lta8i5u6q/ad8145c8-0114-40d4-8e01-3029c65cd6cd/index214.ts
06-17 15:31:58.755  17126-17228/io.kickflip.sample D/HlsFileObserver﹕ Event 8 at index.m3u8.tmp ext tmp
06-17 15:31:58.755  17126-17228/io.kickflip.sample D/HlsFileObserver﹕ Event 128 at index.m3u8 ext m3u8
06-17 15:31:58.764  17126-17228/io.kickflip.sample I/MainActivity﹕ onBroadcastStop
06-17 15:31:58.766  17126-17126/io.kickflip.sample I/CameraEncoder﹕ onHostActivityPaused
06-17 15:31:58.767  17126-17228/io.kickflip.sample I/S3Manager﹕ Queueing upload ies_test/jo2lta8i5u6q/ad8145c8-0114-40d4-8e01-3029c65cd6cd/index.m3u8
06-17 15:31:58.778  17126-17228/io.kickflip.sample I/S3Manager﹕ Queueing upload ies_test/jo2lta8i5u6q/ad8145c8-0114-40d4-8e01-3029c65cd6cd/vod.m3u8
06-17 15:31:58.863  17126-17223/io.kickflip.sample W/FFmpegMuxer﹕ Dropping frame because Muxer not ready!
06-17 15:31:58.863  17126-17223/io.kickflip.sample W/FFmpegMuxer﹕ Dropping frame because Muxer not ready!
06-17 15:31:58.864  17126-17223/io.kickflip.sample W/FFmpegMuxer﹕ Dropping frame because Muxer not ready!
06-17 15:31:59.010  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:31:59.102  17126-17126/io.kickflip.sample I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@3ef42551 time:14529623
06-17 15:31:59.115  17126-17217/io.kickflip.sample I/CameraEncoder﹕ handleRelease
06-17 15:31:59.115  17126-17217/io.kickflip.sample D/Texture2dProgram﹕ deleting program 45
06-17 15:32:00.160  17126-17126/io.kickflip.sample I/API﹕ request succeeded io.kickflip.sdk.api.json.StreamList@3d6fb1f2
06-17 15:32:00.239  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:32:00.645  17126-17126/io.kickflip.sample I/API﹕ request succeeded io.kickflip.sdk.api.json.StreamList@3e788e87
06-17 15:32:00.713  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:32:01.458  17126-17126/io.kickflip.sample I/API﹕ request failed A Kickflip server error occurred
06-17 15:32:02.690  17126-17500/io.kickflip.sample I/S3Manager﹕ Uploaded 2289.276 KB in 3930ms (582.512 KBps)
06-17 15:32:02.701  17126-17498/io.kickflip.sample I/S3Manager﹕ Upload complete.
06-17 15:32:02.883  17126-17500/io.kickflip.sample I/S3Manager﹕ Uploaded 0.244 KB in 178ms (1.37 KBps)
06-17 15:32:02.885  17126-17500/io.kickflip.sample I/MainActivity﹕ onBroadcastLive @ https://kickflip.io/ad8145c8-0114-40d4-8e01-3029c65cd6cd
06-17 15:32:02.887  17126-17498/io.kickflip.sample I/S3Manager﹕ Upload complete.
06-17 15:32:03.084  17126-17500/io.kickflip.sample I/S3Manager﹕ Uploaded 6.582 KB in 194ms (33.927 KBps)
06-17 15:32:03.096  17126-17498/io.kickflip.sample I/S3Manager﹕ Last Upload complete.
06-17 15:32:03.096  17126-17498/io.kickflip.sample I/S3Manager﹕ Shutting down
06-17 15:34:11.020  17126-17136/io.kickflip.sample I/art﹕ Debugger is no longer active
06-17 15:35:22.067  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:35:23.288  17126-17136/io.kickflip.sample W/art﹕ Suspending all threads took: 6.570ms
06-17 15:35:23.340  17126-17126/io.kickflip.sample I/API﹕ request succeeded io.kickflip.sdk.api.json.StreamList@c9cf43
06-17 15:35:23.417  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:35:23.427  17126-17126/io.kickflip.sample I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@3ef42551 time:14733948
06-17 15:35:23.802  17126-17126/io.kickflip.sample I/API﹕ request succeeded io.kickflip.sdk.api.json.StreamList@156193aa
06-17 15:35:23.873  17126-17126/io.kickflip.sample I/StreamListFragment﹕ Loading more streams
06-17 15:35:23.889  17126-17126/io.kickflip.sample I/Muxer﹕ Created muxer for output: /storage/emulated/0/MySampleApp/107335b8-cd8b-4e79-b06e-4078dc43b2b3/index.m3u8
06-17 15:35:23.897  17126-17126/io.kickflip.sample I/Timeline﹕ Timeline: Activity_launch_request id:io.kickflip.sample time:14734418
06-17 15:35:24.001  17126-17126/io.kickflip.sample I/BroadcastFragment﹕ Recreating BroadcastFragment
06-17 15:35:24.013  17126-13298/io.kickflip.sample I/OMXClient﹕ Using client-side OMX mux.
06-17 15:35:24.031  17126-13298/io.kickflip.sample E/OMXMaster﹕ A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one.
06-17 15:35:24.035  17126-13298/io.kickflip.sample D/MediaCodec﹕ MediaCodec[kWhatConfigure]: video-output-protection: 00000000, audio-output-protection: 00000000
06-17 15:35:24.054  17126-17126/io.kickflip.sample I/Kickflip﹕ Clearing SessionConfig
06-17 15:35:24.067  17126-17126/io.kickflip.sample W/CameraRelease﹕ Didn't try to open camera onHAResume. rec: false mSurfaceTexture ready?  no
06-17 15:35:24.188  17126-13307/io.kickflip.sample D/CameraSurfaceRenderer﹕ onSurfaceCreated
06-17 15:35:24.193  17126-13307/io.kickflip.sample D/Texture2dProgram﹕ Created program 3 (TEXTURE_EXT)
06-17 15:35:24.197  17126-13307/io.kickflip.sample D/Texture2dProgram﹕ Created program 6 (TEXTURE_2D)
06-17 15:35:24.199  17126-13307/io.kickflip.sample D/CameraSurfaceRenderer﹕ onSurfaceChanged 1920x1200
06-17 15:35:24.199  17126-13307/io.kickflip.sample I/CameraSurfaceRenderer﹕ setTexSize on display Texture
06-17 15:35:24.203  17126-13325/io.kickflip.sample I/OMXClient﹕ Using client-side OMX mux.
06-17 15:35:24.229  17126-17126/io.kickflip.sample I/API﹕ request failed A Kickflip server error occurred
06-17 15:35:24.248  17126-17126/io.kickflip.sample I/Timeline﹕ Timeline: Activity_idle id: android.os.BinderProxy@33c013e5 time:14734769
06-17 15:35:24.290  17126-13324/io.kickflip.sample D/MediaCodec﹕ MediaCodec[kWhatConfigure]: video-output-protection: 00000000, audio-output-protection: 00000000
06-17 15:35:24.292  17126-13325/io.kickflip.sample E/ACodec﹕ [OMX.qcom.video.encoder.avc] storeMetaDataInBuffers (output) failed w/ err -2147483648
06-17 15:35:24.293  17126-13325/io.kickflip.sample W/ACodec﹕ do not know color format 0x7fa30c04 = 2141391876
06-17 15:35:24.293  17126-13325/io.kickflip.sample W/ACodec﹕ do not know color format 0x7f000789 = 2130708361
06-17 15:35:24.301  17126-13325/io.kickflip.sample I/ACodec﹕ [OMX.qcom.video.encoder.avc] setupVideoEncoder succeeded
06-17 15:35:24.302  17126-13325/io.kickflip.sample W/ACodec﹕ do not know color format 0x7f000789 = 2130708361
06-17 15:35:24.303  17126-13325/io.kickflip.sample D/ACodec﹕ Found video-output-protection flags set to 00000000

This always happened during writing of index214.ts (approximately 1.5 GB of data written), on Sony SGP611 5.0.2, Galaxy Camera 2 EK-GC200 with 4.3 and HTC ONE M7 5.0.2.

Hope this is enough, I can provide more info on different devices if you need, just tell me. Keep up the good work.

P.S. I removed most of the FFmpegWrapper because they added nothing useful and made the logcat less readable.

Arlem commented 9 years ago

Just a small update, I have made more tests and can now exclude that the size of the data written (1.5GB) or the number of .ts files are the culprit; all registration stopped at approximately 40 minutes, as before. Since it is a problem with FFMPEG, I will try to compile the newer version and go ahead from there.

Arlem commented 9 years ago

I updated FFMPEG to version 2.7.1, no differences encountered.

Arlem commented 9 years ago

@OnlyInAmerica Have you got a chance to look into this?

gouravd commented 9 years ago

@Arlem I am having a hard time building FFMPEG 2.7.1 with rtmp. WOuld you be able to help? I want to use the native rtmp instead of librtmp.

I am able to separately build ffmpeg, openssl (using a different method than guardianproject's). I am not able to build librtmp. I am looking for a script that can build librtmp with my generated openssl. Any help would be appreciated

JajuanWang commented 9 years ago

@Arlem I have also encountered this problem.Stoped at approximately 35 minutes every time.To reproduce the bug running at least 35 minutes. Have you solved it? Looking forward to your reply,thanks

JajuanWang commented 9 years ago

Working properly before approximately 35 minutes, but after about 35 minutes it stop recording and log that: 08-27 23:58:05.158: E/FFmpegWrapper(28230): av_interleaved_write_frame video: 1 pkt: 60493 size: 8892 error: Invalid argument

Arlem commented 9 years ago

@gouravd Sorry, I am using hls and haven't tried to build FFMPEG 2.7.1 with rtmp. @JajuanWang No solution found, do you have any guess?

JajuanWang commented 8 years ago

@Arlem https://github.com/OpenWatch/FFmpegWrapper-Android/blob/master/FFmpegWrapper/jni/FFmpegWrapper.c on line 413 //packet->pts = av_rescale_q(packet->pts, *videoSourceTimeBase, (outputFormatContext->streams[packet->stream_index]->time_base));I was going to comment this line,because i need not to adjust the time base,I guess the problem is caused by the this.I have tried to verify it,but recompile the .so files failed

JajuanWang commented 8 years ago

@Arlem and you can see the link http://www.ffmpeg.org/doxygen/2.0/mathematics_8c_source.html on line 130 int64_t av_rescale_q(int64_t a, AVRational bq, AVRational cq),I just known it is used to adjust the time base

Arlem commented 8 years ago

@JajuanWang It fails when you run ndk-build.sh or build_ffmpeg_for_android.sh?

Arlem commented 8 years ago

@JajuanWang I solved (this) problem! Open FFmpegWrapper.c, go to line 420 and modify the code from:

packet->pts = (int) jPts;

to:

packet->pts = (int64_) jPts;

Then, recompile the FFmpeg library and wrapper with

sudo ndk-build.sh

and copy the generated library in the correct folder of your project. I've done registration for more than 1 hour with this change. I have checked that the error originates from jPts being a jlong, which overflows when casted to int (at around 35-40 minutes). I don't know why it was casted to int, since pts is int64_t (as described by https://ffmpeg.org/doxygen/2.8/structAVPacket.html).

dyoungjr08 commented 8 years ago

I have FFmpegWrapper.so and cant view or edit it. what would be the solution in this case. Thank you.

dyoungjr08 commented 8 years ago

@Arlem can you please share the corrected and updated. .so files. Thank you

Arlem commented 8 years ago

@dyoungjr08 This is my .so file of the FFmpeg wrapper (for armeabi and armeabi-v7a). libFFmpegWrapper.zip

Tell me if you need the other FFmpeg library also.

By the way, have you seen this project? https://github.com/OnlyInAmerica/FFmpeg-Android you might need to recompile on your own in the future.

dyoungjr08 commented 8 years ago

Thank you. I believe I do need the others. Im getting this crash now when I try and start a stream:

04-13 22:43:33.562 25102-25102/com.peekatucorp.thePartyStarter I/Muxer: Created muxer for output: /storage/emulated/0/MySampleApp/1fd9e0d2-92de-45b1-883f-192b7a2554a0/index.m3u8 04-13 22:43:33.564 25102-25102/com.peekatucorp.thePartyStarter E/art: dlopen("/data/app/com.peekatucorp.thePartyStarter-1/lib/arm/libFFmpegWrapper.so", RTLD_LAZY) failed: dlopen failed: library "libavformat-57.so" not found 04-13 22:43:33.564 25102-25102/com.peekatucorp.thePartyStarter D/AndroidRuntime: Shutting down VM 04-13 22:43:33.565 25102-25102/com.peekatucorp.thePartyStarter E/AndroidRuntime: FATAL EXCEPTION: main Process: com.peekatucorp.thePartyStarter, PID: 25102 java.lang.UnsatisfiedLinkError: dlopen failed: library "libavformat-57.so" not found at java.lang.Runtime.loadLibrary(Runtime.java:371) at java.lang.System.loadLibrary(System.java:988) at net.openwatch.ffmpegwrapper.FFmpegWrapper.<clinit>(FFmpegWrapper.java:43) at io.kickflip.sdk.av.FFmpegMuxer.<init>(FFmpegMuxer.java:72) at io.kickflip.sdk.av.FFmpegMuxer.create(FFmpegMuxer.java:101) at io.kickflip.sdk.av.SessionConfig$Builder.<init>(SessionConfig.java:225) at com.peekatucorp.io.kickflip.sample.Util.create720pSessionConfig(Util.java:58) at com.peekatucorp.thePartyStarter.MainActivity.configureNewBroadcast(MainActivity.java:2141) at com.peekatucorp.thePartyStarter.MainActivity.startBroadcastingActivity(MainActivity.java:2134) at com.peekatucorp.io.kickflip.sample.fragment.StreamListFragment$4.onClick(StreamListFragment.java:241) at android.view.View.performClick(View.java:4786) at android.view.View$PerformClick.run(View.java:19897) at android.os.Handler.handleCallback(Handler.java:739) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:135) at android.app.ActivityThread.main(ActivityThread.java:5300) at java.lang.reflect.Method.invoke(Native Method) at java.lang.reflect.Method.invoke(Method.java:372) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:904) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:699) 04-13 22:43:33.566 25102-25102/com.peekatucorp.thePartyStarter D/AppTracker: App Event: crash

Arlem commented 8 years ago

@dyoungjr08 These are the ones I am actually using. Let me know if they work ffmpeg+wrapper.zip