search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.41k stars 2.55k forks source link

Question about removing remuxEmptyAudio #6528

Open raxxarr opened 2 days ago

raxxarr commented 2 days ago

What do you want to do with Hls.js?

Hi, I notice that there is a workaround has been removed by this commit, but i cannot fully understand the descripiton: "Remove empty audio track workaround because it causes gaps in live streams with audio PES carryover", could you please help to explain further? Because this workaround could resolve a problem caused by my broken vod stream: the audio data suddenly disappears after a certain fragment. If I added this workaround back in my code, what potential risks might there be？ Thank you.

What have you tried so far?

No response

robwalch commented 2 days ago

In muxed audio video segments audio samples ending before video, this call to remuxEmptyAudio could insert silent audio that may be carried in the next segment.

Silent audio insertion is now only handled in remuxAudio.

raxxarr commented 1 day ago

Thanks for the answer, now I understand why it's removed.

If there will be no more audio data in the rest of the stream, remuxAudio will not handle this, I think it will only insert silent frames when met large audio PTS delta.

I know this is actually a problem of the stream, but is there any other way to make a workaround? for example, still use remuxEmptyAudio, but check overlap and drop some silent frames when next segment arrived?

robwalch commented 1 day ago

You could get the audio SourceBuffer from BUFFER_CREATED, and try appending silent audio after FRAG_BUFFERED based on fragments.elementaryStreams.