Prepare second decoder in advance to avoid buffering at end of ad breaks

[Hiwensen]

Version

ExoPlayer 2.19.1

More version details

It can also be reduced on ExoPlayer 2.18.5.

Devices that reproduce the issue

Not device-specific.

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

1. Add the sample below to the media.exolist.json file

   {
       "name": "VMAP three empty midrolls, player buffer at ad points",
       "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mkv/android-screens-lavf-56.36.100-aac-avc-main-1280x720.mkv",
       "ad_tag_uri": "https://run.mocky.io/v3/d3c23db8-aaf8-43a4-9690-3398a0f0f10f"
     }

2. Add log to Player.onPlaybackStateChanged(@Player.State int playbackState) callback at PlayerActivity.PlayerEventListener
3. Play the added sample video without seeking.

This issue can also be reproduced with the sample VMAP full, empty, full midrolls.

Expected result

The ad point plays seamlessly when there are no ads, and there is no lag. At least it shouldn’t be lag every time.

Actual result

There is almost always an onPlaybackStateChanged buffer callback at the first(10s) and third(30s) ad point, which causes the playback to freeze briefly(around 1s). The second ad point(20s) also has a buffer callback sometimes.

Media

https://storage.googleapis.com/exoplayer-test-media-1/mkv/android-screens-lavf-56.36.100-aac-avc-main-1280x720.mkv

Bug Report

• [X] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[Hiwensen]

Below are the two screen-recording videos. The playbackState always has a buffer at the first and third cue points, whether ad-empty or ad-filled. The buffer happens right after the ad playback when ad-filled.

https://github.com/androidx/media/assets/14145498/ece77d03-bae8-45d5-8cf7-55a96fe9c79c

https://github.com/androidx/media/assets/14145498/2fd2ecf5-1d73-42b2-a0fb-fad302eaa62e

[Hiwensen]

Context I'm following this doc(https://developer.android.com/guide/topics/media/exoplayer/ad-insertion#client-using-third-party) implementing a custom AdsLoader that wraps our ads SDK. We use our ad server to play real-time bidding ads. We will pre-set some ad points and only know whether ads are filled after requesting the ad server a few seconds before the ad points(sometimes the ad point was filled with ads, sometimes the ad points are empty), and meet this issue. I found this issue can also be reproduced in the demo app with IMA SDK applying the same scenario, then raised this issue.

[tonihei]

I think is this working as expected because the ad insertion point is not at a video keyframe and the content piece after the ad has to re-decode everything from the previous keyframe before continuing. In the short time while we wait for this to happen, the player enters STATE_BUFFERING to reflect that it is not making progress.

[tonihei]

For all empty ad points, the player plays through in all tests I've done and this is also working as expected. I can't reproduce the buffering you see at this point, but I assume it might happen if we realize too late that the ad group is empty (the ads SDK is too slow).

[Hiwensen]

@tonihei Thank you for getting back to me. Is there any way to improve it? This playback experience could be better.

I am looking at the source code trying to improve this experience, but the efficiency is a bit slow, and there is a lot of related code. Can you give some tips on which classes or methods to start with?

[tonihei]

Starting from a non-keyframe always requires re-decoding, so the only way to make this perfectly smooth is to use a second decoder that can be prepared in parallel. We are already working on some experimental changes to allow such behavior, but it's fairly complicated and can't be easily added by an app at the moment. Will keep it open as a feature request for now.

[Hiwensen]

Starting from a non-keyframe always requires re-decoding, so the only way to make this perfectly smooth is to use a second decoder that can be prepared in parallel. We are already working on some experimental changes to allow such behavior, but it's fairly complicated and can't be easily added by an app at the moment. Will keep it open as a feature request for now.

Make sense, thank you!