Issue with Surface Handling when Switching Between Two ExoPlayers

[Ioan1995]

I am implementing a scenario where I use two ExoPlayers to play different content on the same Surface, switching between them. The first player is used for the main content, and the second player provides explanations for certain terms. Here's the sequence of operations I follow:

Initialize and start the first player on a Surface. Pause the first player and detach the Surface. Initialize and start the second player on the same Surface. When the second player finishes, detach the Surface and reattach it to the first player. Resume the first player from where it left off. Sometimes, after switching back to the first player, the video remains stuck on the initial frame, although the audio continues to play correctly.

Here's the code I am using:

// Initializing ExoPlayers
SimpleExoPlayer player1 = new SimpleExoPlayer.Builder(context).build();
SimpleExoPlayer player2 = new SimpleExoPlayer.Builder(context).build();

SurfaceView surfaceView = findViewById(R.id.surface_view);
Surface surface = surfaceView.getHolder().getSurface();

// Configuring the first player
player1.setVideoSurface(surface);
player1.setMediaItem(MediaItem.fromUri("first_url"));
player1.prepare();
player1.play();

// Pause the first player and switch to the second player
player1.pause();
player1.setVideoSurface(null); // Remove Surface

// Configuring the second player
player2.setVideoSurface(surface);
player2.setMediaItem(MediaItem.fromUri("second_url"));
player2.prepare();
player2.play();

// Handling end of playback for the second player
player2.addListener(new Player.Listener() {
    @Override
    public void onPlaybackStateChanged(int state) {
        if (state == Player.STATE_ENDED) {
            player2.setVideoSurface(null); // Remove Surface from the second player

            // Reassign the Surface to the first player and resume playback
            player1.setVideoSurface(surface);
            player1.play();
        }
    }
});

I am encountering an issue where, after the second player finishes and the Surface is reassigned to the first player, the video remains stuck on the initial frame while the audio plays correctly.

Additional Information: If I call stop() and prepare() on the first ExoPlayer before resuming playback, it works correctly. However, I am looking for a more efficient way to handle this without stopping and preparing the player again.

Can you please help me identify what might be causing this issue and how to resolve it?

Steps to Reproduce:

Initialize and start the first player with video content. Pause the first player and detach the Surface. Initialize and start the second player on the same Surface. Let the second player play to the end and then switch back to the first player.

Thank you for your assistance!

[microkatz]

Hello @Ioan1995,

When setting an output surface, that is not only setting a resource for rendering the frames, but it is also setting a resource used by the decoder.

https://developer.android.com/reference/android/media/MediaCodec#setOutputSurface(android.view.Surface)

When one sets the outputSurface to null for player 1, MediaCodecVideoRenderer actually releases the decoder and reinitializes it(using a placeholder surface to continue decoding). https://github.com/google/ExoPlayer/blob/release-v2/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java#L711.

The release may flush and drop certain frames. You said only sometimes it fails to play? There may be some disconnect between the current playback position and the next provided frames to be released.

Suggestions:

• You can stop() and prepare() earlier than the moment that the second player needs to finish. Then just call player1.setVideoSurface(surface); and player1.play() at the transition point as you do.
• Try seeking instead of stop() and prepare(). That may be quicker
• You can use two different surfaces/views and just mask one depending on which player needs to play

Hope this helps!

[Ioan1995]

Hello @microkatz,

I saw the suggestions. The first one would work if the user couldn't go back (so it's not a solution because I don't know when they navigate back).

I tested with seek, but it only works after 2-3 seconds after the stream started; otherwise, nothing happens, and I am left with a black screen.

And regarding point 3, that's where I started with having 2 surfaces, but I still end up with a black screen.

I will try to read a bit of the documentation on Surface and look into the code. Maybe I can create a scenario to reproduce it in a demo.I will get back as soon as possible with a log.

[microkatz]

Hello @Ioan1995,

Hopefully you were able to make some progress. Recently, we updated our demo ui code to include an additional ImageView for Image playback. Maybe this commit can provide you with some additional help or inspiration for the UI elements, https://github.com/androidx/media/commit/4cd8d64446724e5301b2b6471a05fa2fa3f56b5e. When general video playback occurs, then the imageView is set to hidden. If image playback is occurring, then it sets the image view to invisible.

Hopefully this helps you.