search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.4k stars 2.55k forks source link

Low Latency playback - recovering after stalls #6350

Open vk342 opened 2 months ago

vk342 commented 2 months ago

What do you want to do with Hls.js?

I am trying to support ultra low-latency HLS stream using 6 second segments and 0.5 second parts with liveSyncDuration of 1.4 seconds.

Hls.js version is 1.5.7.

My config looks as following:

{
    "debug": false,
    "enableWorker": true,
    "lowLatencyMode": true,
    "backBufferLength": 90,
    "liveSyncDuration": 1.4,
    "liveMaxLatencyDuration": 5,
    "maxLiveSyncPlaybackRate": 1.2,
    "liveDurationInfinity": true,
    "startFragPrefetch": true
}

Periodically the player encounters a stall, and after each stall LatencyController target latency is increased by 1 second: this.stallCount * liveSyncOnStallIncrease, see targetLatency below: https://github.com/video-dev/hls.js/blob/0a497317de8e73e6ad0f35bc06c8b327e10c995f/src/controller/latency-controller.ts#L42-L71

So if the stream is played for an hour and encountered 3 stalls our target latency drifts from 1.4 to 4.4.

I might be missing something, but I don't seem to see a mechanism to recover from this condition.

I do not see the API to programmatically change targetLatency after player has been created, so the only way to return to lower latency is to reset the player.

I am looking for a more graceful way to return to lower targetLatency after a period of successful playback.

What have you tried so far?

I have tried to programmatically set LatencyController.stallCount to a lower value after X seconds of playback without a stall. https://github.com/video-dev/hls.js/blob/0a497317de8e73e6ad0f35bc06c8b327e10c995f/src/controller/latency-controller.ts#L19 This did reduce player.targetLatency, but did not affect the actual playback latency, despite the fact that maxLiveSyncPlaybackRate is set to 1.2 and I was hoping that the player would try to catch up to targetLatency.

I have also tried to change liveSyncOnStallIncrease from 1.0 to 0.1 second. https://github.com/video-dev/hls.js/blob/0a497317de8e73e6ad0f35bc06c8b327e10c995f/src/controller/latency-controller.ts#L63 This did cause targetLatency to drift slower, however I still observed that the actual playback latency remained higher then the target latency and the player did not attempt to catch up.

Note: whenever the drift happened, I was able to start another playback in the same stream in a different window using the same version of hls.js and the second player would start and play the stream at my initial target latency of 1.4. For example, my first player would have targetLatency = 1.4, but actual latency of 3.4 sec, while my second player would have both target and actual latency of 1.4 sec.

This indicates that the content has been available on the server.

I would be very grateful for the guidance on how to improve the playback latency in this scenario.

robwalch commented 2 months ago

While stalling might be inevitable, support for preload hints will help reduce stalls. Ask @iamboorrito to rebase and open a PR for https://github.com/video-dev/hls.js/tree/feature/preload-hint (resolves #3988).

robwalch commented 2 months ago

Not a Contribution

You can find Issues and PRs for Low-Latency on this project board https://github.com/orgs/video-dev/projects/3. You might consider commenting on #4681 as these issues are somewhat related:

robwalch commented 2 months ago

My config looks as following:

I would consider (and test) disabling/dropping "enableWorker": true and "startFragPrefetch": true. There is likely no efficiency gained by using a worker with such small parts (you are adding async messaging for short processes that in this case probably slow down buffering of loaded parts). startFragPrefetch is really intended for VOD. Maybe we can make sure it only applies to init segments for live - that would be a good enhancement request to file if you notice a difference. The problem with using it for live is that you might preload a segment that is never used.

Periodically the player encounters a stall, and after each stall LatencyController target latency is increased by 1 second: this.stallCount * liveSyncOnStallIncrease, see targetLatency below: ... So if the stream is played for an hour and encountered 3 stalls our target latency drifts from 1.4 to 4.4. I might be missing something, but I don't seem to see a mechanism to recover from this condition.

Correct.

I am looking for a more graceful way to return to lower targetLatency after a period of successful playback.

There is no mechanism to reset or reduce the stallCount and its impact on "target latency" over time. Would you consider contributing one?

I have tried to programmatically set LatencyController.stallCount to a lower value after X seconds of playback without a stall.

You are on the right track. Being able to recover or reset target latency would be the first step.

I have also tried to change liveSyncOnStallIncrease from 1.0 to 0.1 second.

liveSyncOnStallIncrease could be a configuration option. We'd accept such a contribution if you would be generous enough to make one.

robwalch commented 2 months ago

Not a Contribution

This did reduce player.targetLatency, but did not affect the actual playback latency

In general HLS.js tries to avoid any direct control of playback - it never calls play() or pause() and only sets currentTime when necessary to start or recover playback. The point is that control of playback (and latency) can be performed using the media element provided to HLS.js. If you think HLS.js should handle this, please submit a contribution that improves or adds more configuration options to the current behavior.

vk342 commented 1 month ago

Thank you so much @robwalch, for the thorough and thoughtful reply, and I apology for such a long delay in reacting.

I have tried the following:

As a next step, I will try to see if some suggestions for https://github.com/video-dev/hls.js/issues/4681 could be implemented (by a newbie) and improve LL stall behaviour.