ABR stuck on lower resolution variant with multi-period DASH

[willdharris]

Have you read the FAQ and checked for duplicate open issues? yes

If the problem is related to FairPlay, have you read the tutorial?

n/a

What version of Shaka Player are you using?

4.10.4 and main

Can you reproduce the issue with our latest release version? yes

Can you reproduce the issue with the latest code from main? yes

Are you using the demo app or your own custom app? custom

If custom app, can you reproduce the issue using our demo app? yes

What browser and OS are you using? LG TV WebOS Samsung Tizen

For embedded devices (smart TVs, etc.), what model and firmware version are you using? LG UQ7590PUB fw: 03.34.00 Samsung UN55TU7000FXZA fw: T-KSU2EAKUC-2211.0

What are the manifest and license server URIs?

can provide via email

What configuration are you using? What is the output of player.getConfiguration()?

default apart from: preferredAudioCodecs = ['mp4a']

What did you do?

Play a DAI multi-period DASH stream with varying aspect ratios and bandwidths between ad and content periods.

What did you expect to happen? Playback of the highest bandwidth and resolution representation - 1920x800 @ 5.7mbps.

What actually happened?

ABR sticks at the 960x400 @ 1.9mbps variant and never switches to a higher resolution.

Are you planning send a PR to fix it? Possibly. Need advice on where this should be fixed.

Further detail

During period combining the representations from the content period all get matched to a respective representation from the first pre-roll period except for the 960x400 @ 1.9mbps variant.

This creates a variant track list like this. Notice the 960x400 @ 1.9mbps variant is actually the highest bandwidth variant in the ladder. That explains why ABR sticks at that variant since it has the highest bandwidth.

This happens because the bandwidth assigned to each variant comes from the bandwidth found in the first period. See PeriodCombiner.concatenateStreams_. So even though in a later period the 1920x1080 variant will have a bandwidth of 5.7mbps, it's assigned bandwidth is only 0.9mbps.

Possible solutions

1. Assign Highest Bandwidth One solution could be to assign the highest bandwidth found in the matched representations to the variant. That would create a ladder like this. Bandwidths increase in step with resolutions and report the highest bandwidth a variant would need for the presentation.

2. Update representation matching Another solution could be to exclude the 960x400 representation found in the content period during stream matching. It seems like this representation could be matched to the 960x540 from the pre-roll period. In that case the variant ladder would look like this. Notice all bandwidths are less than 1mbps. While a fix in stream matching would achieve 1920x1080 playback, it seems like this ladder, with such low bandwidths assigned, could lead to buffering issues during higher bandwidth periods. For example, the ABR manager sees it only needs less than a 1mbps connection to play 1920x1080 @ 0.7mbps. If the client has a 2mbps connection, that would be fine during the first period, but during content periods where the 1920x1080 variant has a 5.7mbps bandwidth, playback would choke trying to play 5mbps content on a 2mbps connection.

Assigning the highest bandwidth to the variant would be a simple fix and seems like the better approach, as it would more accurately inform the ABR manager. Am I overlooking potential issues there? Is there an alternative solution?

[avelad]

@willdharris maybe related to https://github.com/shaka-project/shaka-player/issues/3816 ? What do you think?

@caridley any opinion about the potential fix?

[willdharris]

Oh yeah, https://github.com/shaka-project/shaka-player/issues/3816 describes exactly what we are seeing. I think that proposal would resolve this issue.

[avelad]

@willdharris are you interested on implement it?

[willdharris]

Sure, I can work on that.

[avelad]

I upload a patch that I think could work in this case, if you agree, create a PR with this patch and a test that covers your use case. Thank you so much! prioritizeBitrate.patch

diff --git a/lib/util/periods.js b/lib/util/periods.js
index 57c9a20ad..d9ec6fef1 100644
--- a/lib/util/periods.js
+++ b/lib/util/periods.js
@@ -821,7 +821,7 @@ shaka.util.PeriodCombiner = class {
    * @private
    */
   static concatenateStreams_(output, input) {
-    // We keep the original stream's bandwidth, resolution, frame rate,
+    // We keep the original stream's resolution, frame rate,
     // sample rate, and channel count to ensure that it's properly
     // matched with similar content in other periods further down
     // the line.
@@ -879,6 +879,11 @@ shaka.util.PeriodCombiner = class {
       }
     }

+    // Prioritize the highest bandwidth
+    if (output.bandwidth && input.bandwidth) {
+      output.bandwidth = Math.max(output.bandwidth, input.bandwidth);
+    }
+
     // Combine trick-play video streams, if present.
     if (input.trickModeVideo) {
       if (!output.trickModeVideo) {

[willdharris]

@avelad The patch solves the issue. Thanks! I'll work on a test for this.