search

google / ExoPlayer

This project is deprecated and stale. The latest ExoPlayer code is available in https://github.com/androidx/media
https://developer.android.com/media/media3/exoplayer
Apache License 2.0
21.7k stars 6.02k forks source link

ExoPlayer freezes with RTMP and Insta360 Pro #11157

Closed olijuseju closed 1 year ago

olijuseju commented 1 year ago

Bug

Hi, I'm trying to use Exoplayer in my Android app to watch the streaming that my Insta360 Pro camera transmits. This camera uses RTMP to stream the video in 360 degree. I implemented a simpleExoplayer and now I can watch videos from the camera, but when I try to watch the streaming, the player freezes. Only when I turn off the streaming I can see the last 5 seconds of the streaming and then, it stops. This is the code of the implemented exoPlayer in the onCreate of my activity (The logs are only used to debug and they doesn't display anything in the console) :

    DefaultRenderersFactory renderersFactory;

    DefaultLoadControl loadControl = new DefaultLoadControl.Builder().setBufferDurationsMs(2000, 5000, 1500, 2000).build();
    TrackSelector trackSelector = new DefaultTrackSelector();

    renderersFactory = new DefaultRenderersFactory(this).setExtensionRendererMode( DefaultRenderersFactory.EXTENSION_RENDERER_MODE_PREFER);

    simpleExoPlayer = new SimpleExoPlayer.Builder(context, renderersFactory)
            .setTrackSelector(trackSelector)
            .setLoadControl(loadControl)
            .build();

    simpleExoPlayer.setPauseAtEndOfMediaItems(false);
    simpleExoPlayer.setPlaybackSpeed(1f);
    simpleExoPlayer.setRepeatMode(Player.REPEAT_MODE_OFF);
    simpleExoPlayer.setPlayWhenReady(true);

    simpleExoPlayer.setTrackSelectionParameters(
            simpleExoPlayer.getTrackSelectionParameters()
                    .buildUpon()
                    .setMaxVideoSizeSd()
                    .build()
    );

    simpleExoPlayer.addListener(new Player.Listener() {
        @Override
        public void onPlayerError(PlaybackException error) {
            if (error.errorCode == PlaybackException.ERROR_CODE_UNSPECIFIED) {
                Log.d("ExoplayerError", String.valueOf(error.errorCode));
                // Re-initialize player at the current live window default position.
                simpleExoPlayer.seekToDefaultPosition();
                simpleExoPlayer.prepare();
            } else {
                // Handle other errors.
                Log.d("ExoplayerErrorOtro", String.valueOf(error.errorCode));
            }
        }
    });

    playerView.setPlayer(simpleExoPlayer);

    MediaSource mediaSource =
            new RtspMediaSource.Factory()
                    .setDebugLoggingEnabled(true)
                    .setSocketFactory(SocketFactory.getDefault())
                    .setTimeoutMs(20000)
                    .setForceUseRtpTcp(true)
                    .createMediaSource(MediaItem.fromUri(videoUrl));

    simpleExoPlayer.addMediaSource(mediaSource);

    simpleExoPlayer.prepare();
    simpleExoPlayer.seekTo(0);
microkatz commented 1 year ago

Hi @olijuseju.

Thanks for logging your issue!

First of all, is there a reason that you are using SimpleExoPlayer rather than ExoPlayer? It seems that you are mostly using "Default" pieces like DefaultRenderersFactory and DefaultTrackSelector anyway.

Second, would you be able to provide a bug report so that I can review logs? You will need to add the following line to your code as well.

simpleExoPlayer.addAnalyticsListener(new EventLogger());

If you're unable to share bug reports publicly, please send them to dev.exoplayer@gmail.com using a subject in the format "Issue #11157" Please also update this issue to indicate you've done this. Thanks!

olijuseju commented 1 year ago

Hi, I can't add that line to my project, I had to use this line instead:

simpleExoPlayer.addAnalyticsListener(new EventLogger(trackSelector));

There is not a reason to use SimpleExoplayer, I also used Exoplayer and the results are the same

olijuseju commented 1 year ago

That's how the Log messages look like:


2023-05-12 12:01:01.676 18453-18453/com.example.jjpeajar.jors_1 I/ExoPlayerImpl: Init 56e0f48 [ExoPlayerLib/2.17.1] [eos, 21061119DG, Xiaomi, 30]
2023-05-12 12:01:01.826 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 0
2023-05-12 12:01:01.845 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: DESCRIBE rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 1
2023-05-12 12:01:02.005 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=2 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 2
    Transport: RTP/AVP/TCP;unicast;interleaved=2-3
2023-05-12 12:01:02.209 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=1 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 3
    Session: gO8rWrhq
    Transport: RTP/AVP/TCP;unicast;interleaved=0-1
2023-05-12 12:01:02.227 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: PLAY rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 4
    Session: gO8rWrhq
    Range: npt=0.000-
2023-05-12 12:01:32.245 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 5
    Session: gO8rWrhq
2023-05-12 12:02:02.250 18453-18533/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 6
    Session: gO8rWrhq
2023-05-12 12:02:21.090 18851-18851/com.example.jjpeajar.jors_1 I/ExoPlayerImpl: Init 56e0f48 [ExoPlayerLib/2.17.1] [eos, 21061119DG, Xiaomi, 30]
2023-05-12 12:02:21.226 18851-18968/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 0
2023-05-12 12:02:21.236 18851-18968/com.example.jjpeajar.jors_1 D/RtspClient: DESCRIBE rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 1
2023-05-12 12:02:21.379 18851-18968/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=2 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 2
    Transport: RTP/AVP/TCP;unicast;interleaved=2-3
2023-05-12 12:02:21.498 18851-18968/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=1 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 3
    Session: y8uP1Ja7
    Transport: RTP/AVP/TCP;unicast;interleaved=0-1
2023-05-12 12:02:21.526 18851-18968/com.example.jjpeajar.jors_1 D/RtspClient: PLAY rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 4
    Session: y8uP1Ja7
    Range: npt=0.000-
2023-05-12 12:02:30.182 19091-19091/com.example.jjpeajar.jors_1 I/ExoPlayerImpl: Init 56e0f48 [ExoPlayerLib/2.17.1] [eos, 21061119DG, Xiaomi, 30]
2023-05-12 12:02:30.327 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 0
2023-05-12 12:02:30.347 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: DESCRIBE rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 1
2023-05-12 12:02:30.490 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=2 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 2
    Transport: RTP/AVP/TCP;unicast;interleaved=2-3
2023-05-12 12:02:30.597 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=1 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 3
    Session: SX5IW18A
    Transport: RTP/AVP/TCP;unicast;interleaved=0-1
2023-05-12 12:02:30.614 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: PLAY rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 4
    Session: SX5IW18A
    Range: npt=0.000-
2023-05-12 12:03:00.626 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 5
    Session: SX5IW18A
2023-05-12 12:03:30.627 19091-19166/com.example.jjpeajar.jors_1 D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 6
    Session: SX5IW18A
olijuseju commented 1 year ago

That's what I receive in the Run tab:

D/EventLogger: audioDecoderInitialized [eventTime=0.68, mediaPos=0.00, window=0, period=0, c2.android.aac.decoder]
D/EventLogger: audioInputFormat [eventTime=0.68, mediaPos=0.00, window=0, period=0, id=null, mimeType=audio/mp4a-latm, codecs=mp4a.40.15, channels=2, sample_rate=48000]
D/RtspClient: RTSP/1.0 200 OK
    CSeq: 2
    Server: C++ RTMP Media Server (www.rtmpd.com)
    Session: uiocvqvG
    Transport: RTP/AVP/TCP;unicast;interleaved=2-3
    X-Powered-By: C++ RTMP Media Server (www.rtmpd.com)
D/RtspClient: SETUP rtsp://192.168.1.105/live/live/trackID=1 RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 3
    Session: uiocvqvG
    Transport: RTP/AVP/TCP;unicast;interleaved=0-1
D/SurfaceUtils: connecting to surface 0xb4000071bad59010, reason connectToSurface
I/BufferQueueProducer: [SurfaceTexture-1-25417-0](id:634900000000,api:3,p:25417,c:25417) connect(): api=3 producerControlledByApp=true
I/MediaCodec: [OMX.MTK.VIDEO.DECODER.AVC] setting surface generation to 26027010
D/SurfaceUtils: disconnecting from surface 0xb4000071bad59010, reason connectToSurface(reconnect)
I/BufferQueueProducer: [SurfaceTexture-1-25417-0](id:634900000000,api:3,p:25417,c:25417) disconnect(): api=3
D/SurfaceUtils: connecting to surface 0xb4000071bad59010, reason connectToSurface(reconnect)
I/BufferQueueProducer: [SurfaceTexture-1-25417-0](id:634900000000,api:3,p:25417,c:25417) connect(): api=3 producerControlledByApp=true
W/ACodec: Failed to set ANW_HWComposer to OMX, ignoring (-2147483648)
D/SurfaceUtils: set up nativeWindow 0xb4000071bad59010 for 1440x720, color 0x1000, rotation 0, usage 0x402900
D/SurfaceUtils: disconnecting from surface 0xb4000071bac15010, reason disconnectFromSurface
I/BufferQueueProducer: [SurfaceTexture-1-25417-1](id:634900000001,api:3,p:25417,c:25417) disconnect(): api=3
D/RtspClient: RTSP/1.0 200 OK
    CSeq: 3
    Server: C++ RTMP Media Server (www.rtmpd.com)
    Session: uiocvqvG
    Transport: RTP/AVP/TCP;unicast;interleaved=0-1
    X-Powered-By: C++ RTMP Media Server (www.rtmpd.com)
D/RtspClient: PLAY rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 4
    Session: uiocvqvG
    Range: npt=0.000-
D/RtspClient: RTSP/1.0 200 OK
    CSeq: 4
    Server: C++ RTMP Media Server (www.rtmpd.com)
    Session: uiocvqvG
    X-Powered-By: C++ RTMP Media Server (www.rtmpd.com)
D/CCodecConfig: c2 config diff is   c2::i32 coding.drc.compression-mode.value = 1
D/CCodecBuffers: [c2.android.aac.decoder#462:Output[N]] popFromStashAndRegister: output format changed to AMessage(what = 0x00000000) = {
      int32_t aac-drc-album-mode = 0
      int32_t aac-drc-boost-level = 127
      int32_t aac-drc-cut-level = 127
      int32_t aac-drc-effect-type = 3
      int32_t aac-drc-heavy-compression = 1
      int32_t aac-drc-output-loudness = -1
      int32_t aac-encoded-target-level = -1
      int32_t aac-max-output-channel_count = 8
      int32_t aac-target-ref-level = 64
      int32_t channel-count = 2
      string mime = "audio/raw"
      int32_t sample-rate = 48000
    }
D/EventLogger: videoSize [eventTime=1.45, mediaPos=0.00, window=0, period=0, 1440, 720]
D/EventLogger: renderedFirstFrame [eventTime=1.45, mediaPos=0.00, window=0, period=0, Surface(name=android.graphics.SurfaceTexture@e12c8b8)/@0x4502acd]
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 6(49152 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/462 (fetch/transfer)
    evictor expired: 1, evicted: 1
I/System.out: [socket]:check permission begin!
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/951 (fetch/transfer)
W/RtpH264Reader: Received RTP packet with unexpected sequence number. Expected: 0; received: 1. Dropping packet.
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/1453 (fetch/transfer)
I/System.out: [socket]:check permission begin!
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/1961 (fetch/transfer)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/2455 (fetch/transfer)
I/System.out: [socket]:check permission begin!
D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 5
    Session: uiocvqvG
D/RtspClient: RTSP/1.0 200 OK
    CSeq: 5
    Public: DESCRIBE, OPTIONS, PAUSE, PLAY, SETUP, TEARDOWN, ANNOUNCE, RECORD
    Server: C++ RTMP Media Server (www.rtmpd.com)
    Session: uiocvqvG
    X-Powered-By: C++ RTMP Media Server (www.rtmpd.com)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/2957 (fetch/transfer)
W/jjpeajar.jors_: Accessing hidden method Landroid/media/AudioManager;->getOutputLatency(I)I (greylist, reflection, allowed)
I/jjpeajar.jors_: Background young concurrent copying GC freed 29475(5350KB) AllocSpace objects, 0(0B) LOS objects, 7% free, 58MB/63MB, paused 5.111ms total 85.610ms
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/3463 (fetch/transfer)
D/EventLogger: state [eventTime=43.31, mediaPos=0.00, window=0, period=0, READY]
D/EventLogger: isPlaying [eventTime=43.31, mediaPos=0.00, window=0, period=0, true]
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/3989 (fetch/transfer)
D/EventLogger: state [eventTime=43.33, mediaPos=0.00, window=0, period=0, BUFFERING]
D/EventLogger: isPlaying [eventTime=43.33, mediaPos=0.00, window=0, period=0, false]
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/4495 (fetch/transfer)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/5021 (fetch/transfer)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/5531 (fetch/transfer)
D/RtspClient: OPTIONS rtsp://192.168.1.105/live/live RTSP/1.0
    User-Agent: ExoPlayerLib/2.17.1
    CSeq: 6
    Session: uiocvqvG
D/RtspClient: RTSP/1.0 200 OK
    CSeq: 6
    Public: DESCRIBE, OPTIONS, PAUSE, PLAY, SETUP, TEARDOWN, ANNOUNCE, RECORD
    Server: C++ RTMP Media Server (www.rtmpd.com)
    Session: uiocvqvG
    X-Powered-By: C++ RTMP Media Server (www.rtmpd.com)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/6043 (fetch/transfer)
D/EventLogger: state [eventTime=67.52, mediaPos=0.00, window=0, period=0, READY]
D/EventLogger: isPlaying [eventTime=67.52, mediaPos=0.00, window=0, period=0, true]
D/EventLogger: state [eventTime=67.54, mediaPos=0.00, window=0, period=0, BUFFERING]
D/EventLogger: isPlaying [eventTime=67.55, mediaPos=0.00, window=0, period=0, false]
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/6541 (fetch/transfer)
D/BufferPoolAccessor2.0: bufferpool2 0xb4000071bad20428 : 5(40960 size) total buffers - 5(40960 size) used buffers - 1/7 (recycle/alloc) - 6/7047 (fetch/transfer)
microkatz commented 1 year ago

@olijuseju.

Hi, I can't add that line to my project, I had to use this line instead: simpleExoPlayer.addAnalyticsListener(new EventLogger(trackSelector));

What version of ExoPlayer are you using?

There is not a reason to use SimpleExoplayer, I also used Exoplayer and the results are the same

So you used ExoPlayer.Builder and did not use your custom loadControl as well?

From your logs I can see that the player is stuck in a buffering state and not playing. I'm not sure if that's a result of your custom loadControl settings or that the camera is not sending enough packets for playing to start. Once the buffer is filled to the set minimum then playback would start. It may be that you only have playback when you stop streaming as then ExoPlayer sees that no more data is coming and it must play what it has.

olijuseju commented 1 year ago

Hi, I finally fix the bug. This is the code I use in the onCreate() function:


        AndroidCompat.setSustainedPerformanceMode(this, true);
        AndroidCompat.setVrModeEnabled(this, true);

        playerView = findViewById(R.id.viewD);

        SphericalGLSurfaceView sphericalGLSurfaceView = (SphericalGLSurfaceView) playerView.getVideoSurfaceView();

        sphericalGLSurfaceView.setUseSensorRotation(true);

        //Crear el TrackSelector
        TrackSelector trackSelector = new DefaultTrackSelector();

        //Crear el player
        simpleExoPlayer = new ExoPlayer.Builder(context)
                .setTrackSelector(trackSelector)
                .setRenderersFactory(new DefaultRenderersFactory(context))
                .build();

        DefaultRenderersFactory renderersFactory = new DefaultRenderersFactory(context);

        playerView.setPlayer(simpleExoPlayer);

And This is function is called when the user clicks the streaming button:

` public void playStreaming(String ipCam) {

    RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory();
    // El MediaSource reproduce el video
    MediaSource videoSource = new ProgressiveMediaSource.Factory(rtmpDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.parse(ipCam)));

    MediaSource videoSource2 = new ProgressiveMediaSource.Factory(rtmpDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.parse("ipCam")));

    MergingMediaSource mergedSource = new MergingMediaSource(videoSource, videoSource2);

    // Preparo el player
    simpleExoPlayer.prepare(mergedSource);
    SphericalGLSurfaceView videoSurfaceView = (SphericalGLSurfaceView) playerView.getVideoSurfaceView();
    videoSurfaceView.setDefaultStereoMode(
            C.STEREO_MODE_STEREO_MESH
    );
    //El stream se reproduce cuando está listo
    simpleExoPlayer.setPlayWhenReady(true);
}`

Also is important to call `playerView.onResume();` in the `onResume`
Thanks for everything
microkatz commented 1 year ago

Great! Glad you were able to fix your issue.