search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.83k stars 2.57k forks source link

Playback Issues with Progressive/Fetch Mode in v1.0.0.alpha-5 #3323

Closed softworkz closed 3 years ago

softworkz commented 3 years ago

What version of Hls.js are you using?

https://github.com/video-dev/hls.js/tree/v1.0.0-alpha.5

What browser and OS are you using?

Chrome, Windows

Test stream:

https://github.com/softworkz/hlstestfiles/releases/tag/1.0

Checklist

Steps to reproduce

Playback using these settings:

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "testBandwidth": false,
  "liveBackBufferLength": 90,
  "progressive": true
}

Expected behavior

Smooth playback

Actual behavior

Playback with progressive: true is causing gaps:

image

Those gaps do not occur when progressive mode is disabled. (but I can see segments lengths being resized after load)

softworkz commented 3 years ago

Possible Cause

I'm suspecting that those issues might occur due to an offset between video and audio of about 500ms.

This is legitimate for MPEGTS streams and can often be seen in TV broadcast streams (like the provided sample is originating from).

Please see the following analysis of the segment details from the provided sample:

image

Legend:

In the first three lines, we can see the durations from Container, Audio and Video. The container duration is always larger because it indicates the duration from the first audio or video sample until the last audio or video sample.

In the playlist, I am using the video durations for the EXTINF tags.

That's also a question I have: Is this correct?

zuzzurro commented 3 years ago

(sorry for the unrelated question) what is the tool that you screenshotted above? Seems pretty nice..

robwalch commented 3 years ago

Hi @zuzzurro, that is our demo page. The v1.0.0-alpha.5 demo page is https://hls-js-766b2bcb-f5ae-4e91-837e-1f415155d866.netlify.app/demo/

You can find the list of build and release demo pages at https://github.com/video-dev/hls.js/tree/deployments

zuzzurro commented 3 years ago

Sorry, the comment was not for you but for @softworkz . I like his screenshot. I also agree that the new "timeline" view in the demo page is great and so useful..

robwalch commented 3 years ago

@softworkz progressive is experimental and not recommended for use in production without testing. I recommend looking at the logs for any dropped content and seeing if you can come up with a fix.

softworkz commented 3 years ago

(sorry for the unrelated question) what is the tool that you screenshotted above? Seems pretty nice..

@zuzzurro - This is a private tool that I've done for media analysis including HLS segments. I don't know yet what I'm going to do with it.

softworkz commented 3 years ago

@softworkz progressive is experimental and not recommended for use in production without testing. I recommend looking at the logs for any dropped content and seeing if you can come up with a fix.

OK, but can you give me some advice which duration I should use for the segments in the media playlist:

robwalch commented 3 years ago

Hi @softworkz,

progressive is only useful if you have long segments with multiple GoP (like 10 seconds with groups that could be appended every 2 seconds). At 2-6 second intervals, transmuxing of whole segments is more efficient memory and CPU wise.

Why do you want to use it?

softworkz commented 3 years ago

Why do you want to use it?

To reduce latency and startup time.

softworkz commented 3 years ago

Why do you want to use it?

To reduce latency and startup time.

..which actually works well. It's just that hlsjs merges the segments incorrectly.

robwalch commented 3 years ago

To reduce latency and startup time.

Unless your users have very constrained bandwidth, you won't get much of an improvement in start time. For latency, the real benefit would be with chunk-transfer, but we've shifted focus to support LL-HLS.

The issue with this stream could be a difference in fragment PTS properties or something similar that impacts appending. It could also be an issue with dropping carryover data between progressive buffer flushes. Comparing the timeline model (fragment properties) as well mp4-remuxer behavior with progressive on and off might reveal the why the gaps are being introduced.

softworkz commented 3 years ago

Unless your users have very constrained bandwidth, you won't get much of an improvement in start time.

It's in a LAN, and it's about live TV streams where the segments are starting to get created once the user starts playback. That means that HLSjs can't download the first three segments right away because there are no three segments. That's why progressive mode makes a huge difference in that case.

softworkz commented 3 years ago

BTW: Did you notice the demo stream that I have prepared for this issue: https://github.com/softworkz/hlstestfiles/releases/tag/1.0

robwalch commented 3 years ago

@softworkz these changes fix the issue: https://github.com/video-dev/hls.js/pull/3358/commits/10f2b4272aa3c68d2cb672384aaaf106ea3dbe59

It turns out hls.js was updating SourceBuffer.timestampOffset a bit late and sometimes aborting audio/mpeg appends that it should not have. This even sometimes caused gaps when progressive was disabled (mostly in our automated tests).

Thanks for the test stream. It was a big help in confirming this issue/fix.

softworkz commented 3 years ago

Very first impression: appears to be working perfectly.

Great job! Thank you very much for looking into this. I'll propagate this change to our testing group and let you know about the results.