search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.91k stars 2.58k forks source link

Fill gaps with empty frames or last frame #6518

Open roflcoopter opened 4 months ago

roflcoopter commented 4 months ago

What do you want to do with Hls.js?

I am trying to synchronize multiple HLS streams side by side from security cameras. They are not guaranteed to have media files for every moment in time (one camera might reboot for instance).

Is there a way to make the #EXT-X-GAP tag be treated differently than nudging the stream ahead? I would like to either fill the gaps with black frames or simply pause the stream for the duration of the gap

Is it possible to override the handling of the #EXT-X-GAP in this way?

What have you tried so far?

I have tried adding the #EXT-X-GAP tag but the stream is nudged forward which makes the streams out of sync.

robwalch commented 4 months ago

It would be ideal if HLS.js could play through GAP tags. Video and Audio buffer gaps, where an alternate playlist and non-gap segment could be found, would need to be filled.

There is no simple way to insert a video frame of arbitrary length in MSE. Extending the duration of the last frame or finding an intra-frame candidate for any GAP tag situation would be very difficult.

Perhaps HLS.js could offer an option to not perform seeks over gaps? As a workaround, try listening for BUFFER_SEEK_OVER_HOLE with frag.gap === true and interrupt the seek operation by pausing and performing multiple seeks over the duration of the gap at a regular interval until the playhead makes it past the unbuffered range. If this works we could consider adding a "nudge-through-gaps" mode to handle playback advancement (provided the application or player using HLS.js can deal with the additional HTMLMediaElement playback and seeking events which we could signal via BUFFER_NUDGE_ON_GAP before each step).

The work to support #EXT-X-GAP has been tracked under #2940 if you're interested in background and past conversations on the subject.

roflcoopter commented 4 months ago

Thanks for the quick reply! Understood, yes pausing the stream is an acceptable solution, will see what i can do with BUFFER_SEEK_OVER_HOLE, will report back soon.

roflcoopter commented 2 months ago

Finally got around to picking this up again, sorry for the delay. I got it mostly working, but i would like the video to pause when no new fragments are added to the playlist (for instance when a camera disconnects)

It seems to retry a few times then it seeks back and plays the last segment again in a loop. I tried reading through the API docs but dont fully understand how to achieve that. Setting maxStarvationDelay: 99999999 and maxLoadingDelay: 99999999 seems to work okay but there must be some other way?

I have setup a listener on Hls.ErrorDetails.BUFFER_STALLED_ERROR which pauses the player.