Low-bitrate ABR adaptation not working very well

kmrinmoy07 commented 1 year ago

Version

Media3 1.1.1

More version details

No response

Devices that reproduce the issue

MI 9T Pro running Android 11 Pixel 6 Pro API 34 (Emulator)

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

In the demo app -

Select a 2G/3G network or any network with lower bandwidth, or use Network Speed Booster and select lower bandwidth network
Now open the demo app
Select HLS
Select Apple Multivariant playlist advanced (TS)
Make sure in 'Select Tracks' option, Auto option is selected
Now play the track

Expected result

Exoplayer selects a lower bitrate or lower resolution video and adapt it with lower bandwidth network, and play the video without buffering

Actual result

Exoplayer selects the highest resolution video or highest bitrate audio and keeps buffering

Media

Mailed to android-media-github@google.com

Bug Report

[ ] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

tonihei commented 1 year ago

At the beginning of playback, ExoPlayer has to guess a suitable bitrate for the first format. If you select a 2G network, it should fallback to a very low estimate (you can check with BandwidthMeter.getBitrateEstimate()). Once it starts with a higher bitrate though, it may take a short time to generate new bandwidth measurements and switch down.

Your stream in particular is interesting because the declared bitrates are very low. Even the highest bitrate is only 320kbps, so the network estimate has to go really, really low to get below this point. The initial estimates I mentioned are listed here, and as you can see, even 2G estimates typically exceeds this bitrate.

You can override the initial estimate that gets applied at the start of playback by modifying the settings in DefaultBandwidthMeter.Builder.setInitialBitrateEstimate. There are different overrides to either specify a global estimate, or per country and/or network type.

kmrinmoy07 commented 1 year ago

@tonihei my hls m3u8 master file is an audio track having 5 different variants of audio quality in it from 320kbps to 54kbps.

As you said the Exoplayer guess a suitable bitrate initially for the first time, accordingly in my case for the first time, Exoplayer guesses and selects the lowest variant of the audio quality, i.e. 54kbps and it is fine, i don't have any issue with it or doesn't want to override this.

But after initial first time, then Exoplayer immediately selects 320kbps audio variant all the time even, no matter whatever initial or in middle of playback, irrespective of network bandwidth 2G Network or 5G Network, so my playback keeps on buffering in lower network speed like in 2G. I have used BandwidthMeter.getBitrateEstimate() while in 2G network and i have got really higher value like 990,000 in 2G Network, but I believe 2G network usually has bandwidth less than 120kbps whereas 990,000 is quite quite high, and I expect Exoplayer to select lower audio variant available in the master playlist for 2G network, but Exoplayer still keeps using 320kbps audio variant.

As you have said -

Your stream in particular is interesting because the declared bitrates are very low. Even the highest bitrate is only 320kbps, so the network estimate has to go really, really low to get below this point. The initial estimates I mentioned are listed here, and as you can see, even 2G estimates typically exceeds this bitrate.

So do you mean, is my master playlist's bandwidth and bitrate having a wrong lower value, should it have higher values to work properly with Exoplayer. I have setup my bandwidth with correspondence to my audio's bitrate, for 320000 bandwidth the bitrate is 320kbps, for 160000 bandwidth the bitrate is 160kbps and so on.

I have seen the initial estimate as mentioned in the url you shared, but i can see the bitrates are quite high for the corresponding network types. Is it the case - bitrate and its corresponding network type having different values based on country to country?

I have also posted a question earlier in StackOverflow here you can have a look at the answer (though the answer doesn't solve the issue using exoplayer's built in automatic adaptive track selection) and comments associated with the answer

tonihei commented 1 year ago

but I believe 2G network usually has bandwidth less than 120kbps whereas 990,000 is quite quite high

You're right, this estimate looks technically wrong when considering the official definition of 2G networks. The issue is that many network providers around the world use "2G" to describe various setups that may not strictly speaking be just 2G. These values are based on real measurements from ExoPlayer clients. It's based on the bandwidth estimate measured while these clients are reported to be on a 2G network. This makes sense because it's then more likely that these initial estimates match the actual measurements you may get on such devices.

So do you mean, is my master playlist's bandwidth and bitrate having a wrong lower value

No, your stream is setup correctly. I was just noting that the bitrates are very low in general, which means that the best resolution is still not using a lot of bandwidth and that the estimate from the BandwidthMeter really has to go very low to see a difference.

I have also posted a question earlier in StackOverflow here you can have a look at the answer (though the answer doesn't solve the issue using exoplayer's built in automatic adaptive track selection) and comments associated with the answer

That answer is a bit confusing and technically wrong in many places (example: the initial player setup code instantiates a BandwidthMeter manually, but it isn't actually used to make any adaptive track selection decisions)

There are multiple potential issues here that might cause the behavior you are seeing:

Small transfers like these low-bitrate audio chunks likely overestimate the available bandwidth due to the way the network stack handles connections.
There is an initial threshold of transferring at least 512kB of data, taking at least 2 seconds overall. As long as these thresholds aren't reached (one of them is fine), no estimate will be updated at all.

We recently published an ExperimentalBandwidthMeter class that provides improved algorithms that should help with both these issues.

Could you try to set up your player like this? I would be curious to see if it helps in your example.

new ExoPlayer.Builder(/* context= */ this)
    .setBandwidthMeter(
        new ExperimentalBandwidthMeter.Builder(/* context= */ this)
            .setTimeToFirstByteEstimator(
                new ExponentialWeightedAverageTimeToFirstByteEstimator())
            .setBandwidthEstimator(
                new CombinedParallelSampleBandwidthEstimator.Builder()
                    .setBandwidthStatistic(new SlidingPercentileBandwidthStatistic())
                    .build())
            .build()

Note that there is a known bug (https://github.com/androidx/media/issues/612) that will be fixed soon and you might want to cherry-pick the fix CL or wait for the next release before using this in production code.

kmrinmoy07 commented 1 year ago

It's based on the bandwidth estimate measured while these clients are reported to be on a 2G network. This makes sense because it's then more likely that these initial estimates match the actual measurements you may get on such devices

So do I need to modify or override the initial estimates to work in my case, do i need to do anything so that my network can work in align with the exoplayer's adaptive algorithm, and if the new ExperimentalBandwidthMeter helps in this case then i would not like to change the initial estimates.

I have tried the code that you have given and my code now looks like -

    AudioAttributes audioAttributes = new AudioAttributes.Builder()
            .setUsage(C.USAGE_MEDIA)
            .setContentType(C.AUDIO_CONTENT_TYPE_MUSIC)
            .build();
    AdaptiveTrackSelection.Factory trackSelectionFactory = new AdaptiveTrackSelection.Factory();
    trackSelector = new DefaultTrackSelector(this, trackSelectionFactory);

    exoPlayer = new ExoPlayer.Builder( this)
                    .setTrackSelector(trackSelector)
                    .setBandwidthMeter(
                            new ExperimentalBandwidthMeter.Builder(this)
                                    .setTimeToFirstByteEstimator(
                                            new ExponentialWeightedAverageTimeToFirstByteEstimator())
                                    .setBandwidthEstimator(
                                            new CombinedParallelSampleBandwidthEstimator.Builder()
                                                    .setBandwidthStatistic(new SlidingPercentileBandwidthStatistic())
                                                    .build())
                                    .build()).setAudioAttributes(audioAttributes, true)
                    .setHandleAudioBecomingNoisy(true).build();
    exoPlayer.addListener(listener);

    DataSource.Factory dataSourceFactory = new DefaultDataSource.Factory(this);//.setTransferListener(bandwidthMeter);
    HlsMediaSource audioSource =  new HlsMediaSource.Factory(dataSourceFactory)
            .setLoadErrorHandlingPolicy(new CustomPolicy())
            .createMediaSource(MediaItem.fromUri("https://mydomain/master-playlist.m3u8"));

    exoPlayer.setMediaSource(audioSource);
    exoPlayer.prepare();
    exoPlayer.setPlayWhenReady(true);

But still i can see that Exoplayer is following the earlier behaviour, and not switching to lower bitrate in low bandwidth.

kmrinmoy07 commented 1 year ago

@tonihei have you got any time to look into my last comment

tonihei commented 1 year ago

Your integration looks correct, sorry it didn't help though. Another thing you can try is to use a different network stack (e.g. Cronet), which has different behavior with regards to bandwidth measurements (https://developer.android.com/guide/topics/media/exoplayer/network-stacks).

Other than that, it's hard to pinpoint a particular issue and I can leave this open for now for a more in-depth analysis why this low-bandwdith ABR logic is not working properly.

androidx / media