Buffering after updating AdPlaybackState with ad data

[kotucz]

Version

ExoPlayer 2.16.1

More version details

Our repo forked from the version above

Devices that reproduce the issue

Multiple devices Emulator Medium Phone API 33 Read device Tablet Android 11 Many user devices

Devices that do not reproduce the issue

N/A

Reproducible in the demo app?

No

Reproduction steps

Summary

We are implementing AdsLoader that will provide our ad data to the player Our ad data is not known from the beginning, so we initialize AdPlaybackState with empty ad groups. During content playback we obtain the actual ad data and update the ad group with it before it is reached. After we update the new AdPlaybackState with the actual ad data (time + media uri) we observe the player starts buffering. How can we avoid this buffering?

Details

As ad breaks are not know on initialization so we allocate empty ad groups like this:

val midrollPlaceholders = LongArray(adBreakCount) { Long.MAX_VALUE }
adPlaybackState = AdPlaybackState(adsId, *midrollPlaceholders)

Marking all uninitialized ad groups as skipped

adPlaybackState.withSkippedAdGroup(adGroupIndex)

If the uninitialized ad groups are not marked as skipped, the playback gets stuck.

Later we resolve the ad break data and update upcoming ad group in some time ahead (e.g. one minute):

adPlaybackState.withAdGroupTimeUs(adGroupIndex, adGroupStartTime)
adPlaybackState.withAdCount(adGroupIndex, adCount)
adPlaybackState.withAdUri(adGroupIndex, adIndexInAdGroup, uri)

And updating the adPlaybackState to player:

AdsLoader.EventListener.onAdPlaybackState(adPlaybackState)

After this, we are observing ~1 sec buffering in the player

Player.Listener fun onPlaybackStateChanged(playbackState: Int) // STATE_BUFFERING

when the buffering is done (STATE_READY), content playing continues and the ad group plays once reached

What can we do to avoid this buffering when updating the ad group?

It looks like the player is buffering the content rather than the new ad group, but there is no change expected in the content playback data.

Expected result

There should be no additional buffering of the content when AdPlaybackState is updated The updated ad group is few minutes ahead, so it should have no impact on current content playback.

Actual result

After this, we are observing ~1 sec buffering in the player

What can we do to avoid this buffering when updating the ad group?

It looks like the player is buffering the content rather than the new ad group, but there is no change expected in the content playback data.

Media

The bug was reproducible with any media. E.g. https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_ts/master.m3u8

Bug Report

• [ ] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[marcbaechinger]

Thanks for your report.

Can you add EventLogger with player.addAnalyticsLogger(EventLogger()) then go through that scenario, make a bug report (or capture the logcat of EventLogger) and upload this here?

I think we should see in the logs what is going on. Without that it's just guessing from my side. I hope the logs will add some important information that helps us investigating.

[kotucz]

Attached the logcat with EventLogger

Medium-Phone-API-33-Android-13_2024-11-15_144155-bug-1892.zip

[kotucz]

Screenshot from Charles. Video segments are fetched again. Which is probably causing the buffering.

[marcbaechinger]

Hmmm, sorry. This file is a json file with each line in a json object. Do you have the simple text version of it somehow? That would make it much easier.

{
3068       "header": {
3069         "logLevel": "DEBUG",
3070         "pid": 29630,
3071         "tid": 29630,
3074         "tag": "EventLogger",
3075         "timestamp": {
3076           "seconds": 1731677910,
3077           "nanos": 286000000
3078         }
3079       },
3080       "message": "audioAttributes [eventTime\u003d0.04, mediaPos\u003d0.00, window\u003d0, 3,0,1,1]"
3081     },

[kotucz]

New log in txt. The previous one is export from logcat. Should be possible to import in Android Studio

exoplayer-log-bug-1892.txt

[marcbaechinger]

Thanks!

It's not really clear what happens. I actually doesn't seem to work at all. The player never attempts to play an ad. It is always on the content period and never tries to enqueue an ad period.

From what you describe, the tricky bits are that you change the positions of the ads which can be tricky. I'm not sure how far you got that working and whether it worked in some ways.

Sorry if you already did what I'm going to suggest. Just checking:

I often mix up Ms and Us. So I'd double check this.

As a start I'd suggest to use the real positions from the very beginning. At least until this works. Like instead of setting the position to Integer.MAX_VALUE use the actual position of the add.

Then instead of setting them to SKIPPED I'd just leave then as is. Then they are in state AD_STATE_UNAVAILABLE and the player won't pick them for playback until you transition them to available. When you then use withAdUri(uri), you transition the ad to AD_STATE_AVAILABLE automatically.

I'm also a bit confused by your snippet:

adPlaybackState.withAdGroupTimeUs(adGroupIndex, adGroupStartTime)
adPlaybackState.withAdCount(adGroupIndex, adCount)
adPlaybackState.withAdUri(adGroupIndex, adIndexInAdGroup, uri)

You probably just use this for the post because it should be something like this:

adPlaybackState =
    adPlaybackState
        .withAdGroupTimeUs(adGroupIndex, adGroupStartTime)
        .withAdCount(adGroupIndex, adCount)
        .withAdUri(adGroupIndex, adIndexInAdGroup, uri);

Guess this is an artifact for this post though.

When it worked, then you see an event positionDiscontinuity that goes from the content period to the ad period. The logs show a skip. That's where something got inserted but something went wrong:

 positionDiscontinuity [eventTime=22.67, mediaPos=10.97, window=0, reason=SKIP, PositionInfo:old [mediaItem=0, period=0, pos=10968], PositionInfo:new [mediaItem=0, period=0, pos=0]]

if it works, the new is something like [..., period=-0, adGroupIndex=0, adIndexInAdGroup=0] which shows the player has transitioned to the ad period.

[kotucz]

Hi, thanks for your suggestions, here is an update on what I tried.

The logs I sent were only from the part when we observe the extra buffering - at the moment when we set ad data to and ad group that was marked as previously SKIPPED. Way before the ad group actually starts playing.

The ad playback works as expected for us. Just for clarity: we are checking the time units. Using adPlaybackState as builder pattern as it is immutable.

Few observations

This buffering we are observing happens when we set the ad data for the upcoming ad group and it was previously (or initially) SKIPPED

If there is some AVAILABLE ad group in between current position and the one being set, the buffering does not appear.

If we are updating already AVAILABLE ad group with some other data or change a time, we do not observe buffering. There is a limitation that ad count in a group may not decrease.

What I have tried

Setting ad groups positions at the beginning

Only this alone does not fix the issue. When ad group initially marked as SKIPPED is set with data buffering occurs. Additionally if we are having ad groups times set initially, we sometimes need to mark them as SKIPPED. If ad group has no ad data player becomes stuck on buffering when attempting to play. Also in some scenarios when user seeks we are skipping some ad breaks and later making them available again. Is there maybe some API that can transfer SKIPPED ad group to initial state? That could help I tried using AdPlaybackState.newAdGroup()`` but this adds extra ad group and ad group count may actually not change when updatingAdPlaybackState`.

Not marking the ad groups as SKIPPED initially is helping. Updating just initial ad groups with ad data -> no buffering. There seems to be no difference what the original ad group time was.

So if I keep all ad groups after the content initially (e.g. MAX_LONG) everything seems to be working: Setting ad group data + time has no buffering when it was not initially SKIPPED. Only issue is that in the end content player tries to play the initial unused ad groups and gets stuck on buffering them. When I detect this case I mark those SKIPPED and player can finish playback. This leaves some non elegant play finish handling + some buffering in the end. Note: I detect this end of playback with onPositionDiscontinuity() callback. Maybe there is better option?

[marcbaechinger]

I detect this end of playback with onPositionDiscontinuity() callback

Yes, this is the only callback that reports period transitions. Check oldPosition for adIndex != -1 with 'DISCONTINUITY_REASON_AUTO`.

I think the knowingly working approach is leaving them in AD_STATUS_UNAVAILABLE and then transition them into AVAILABLE. Once skipped, played or failed I wouldn't transition them back.

I tried using AdPlaybackState.newAdGroup()` but this adds extra ad group and ad group count may actually not change when updating AdPlaybackState.

Yes, correct. Aside: I'm working on adding live suppport to AdsMediaSource. I don't have an ETA yet, but this requires that the number of groups can grow. With my current understanding I think it is feasible to change the source to allow appending AdGroups at the end of the existing AdGroups.

MAX_LONG

Are you using your own UI? Doesn't a MAX_LONG show ad markers at the very end of the Timebar? Is this acceptable?

C.TIME_END_OF_STREAM is used for post rolls, so would be an alternative but I figure it would have these ad markers displayed in the PlayerView.

Just one note why this is difficult: AdsMediaSource actually expects that for VOD all ad insertion points (adGroup.timeUs) are known when starting. Specifically the preroll needs to be known, else it starts playing content before the preroll is inserted.

So what AdsMediaSource is designed for is:

1. Get ad data from whereever and then set call updateAdPlayabackState)state) with all ad groups with the timeUs set. This leaves the ad in UNAVAILABLE.
2. Then if needed, start loading things like duration and URI and ad them to the AdPlaybackState.
3. Continue populating ad group by ad group if required.

Obviously the URI must be made available before the current position is at timeUs of the AdGroup . Note that the player also starts preparing even if unavailable and just hangs until the ad data is available. In this situation it would show being in buffering state until the ad URI is provided and the media loaded from that URI.