video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.84k stars 2.57k forks source link

Audio gap on fragment boundary for some streams #791

Closed frankpaolino closed 7 years ago

frankpaolino commented 7 years ago
Environment

Sample 1: http://abcvod.abcnews.com/i/abcnews/2016/10/161028_gma_whitworth_0813_,500,800,1200,1800,2500,3200,4500,.mp4.csmil/master.m3u8?b=500-4500

Sample 2: http://abcvod.abcnews.com/i/abcnews/2016/10/161031_atm_ok_manhunt_,500,800,1200,1800,2500,3200,4500,.mp4.csmil/master.m3u8?b=500-4500

Steps to reproduce

1) Play above streams at hls.js demo page, such as http://dailymotion.github.io/hls.js/demo/?src=http%3A%2F%2Fabcvod.abcnews.com%2Fi%2Fabcnews%2F2016%2F10%2F161028_gma_whitworth_0813_%2C500%2C800%2C1200%2C1800%2C2500%2C3200%2C4500%2C.mp4.csmil%2Fmaster.m3u8%3Fb%3D500-4500&enableStreaming=true&autoRecoverError=true&enableWorker=true&levelCapping=-1&defaultAudioCodec=undefined 2) Observe during bitrate switch at first 10 second boundary, the audio drops briefly Note the "warn" message [warn] > Injecting 57 audio frame @ 10s due to 1331 ms gap. (Second sample stream has smaller gap)

Expected behavior

Audio plays uninterrupted, as in Safari.

Actual behavior

Observe during bitrate switch at first 10 second boundary, the audio drops briefly (0.5 to 1.5 seconds) Note the "warn" message for first stream: [warn] > Injecting 57 audio frame @ 10s due to 1331 ms gap. Note if you play the same bitrate via child playlist, the audio gap does not occur at all.

The warning message implies there is a gap in the stream, however, then how does it play back uninterrupted in Safari?

Console output
[log] > trigger BUFFER_RESET
logger.js:39 [log] > set autoLevelCapping:-1
logger.js:39 [log] > attachMedia
logger.js:39 [log] > media source opened
logger.js:39 [log] > manifest loaded,6 level(s) found, first bitrate:524000
logger.js:39 [log] > startLoad
logger.js:39 [log] > engine state transition from undefined to STOPPED
logger.js:39 [log] > switching to level 0
logger.js:39 [log] > loading playlist for level 0
logger.js:39 [log] > engine state transition from STOPPED to IDLE
logger.js:39 [log] > engine state transition from IDLE to WAITING_LEVEL
logger.js:39 [log] > audio tracks updated
logger.js:39 [log] > level 0 loaded [1,19],duration:187.82
logger.js:39 [log] > engine state transition from WAITING_LEVEL to IDLE
logger.js:39 [log] > Loading 1 of [1 ,19],level 0, currentTime:0.000,bufferEnd:0.000
logger.js:39 [log] > engine state transition from IDLE to FRAG_LOADING
logger.js:39 [log] > Loaded  1 of [1 ,19],level 0
logger.js:39 [log] > engine state transition from FRAG_LOADING to PARSING
logger.js:39 [log] > Parsing 1 of [1 ,19],level 0, cc 0
logger.js:39 [log] > demuxing in webworker
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > level switch detected
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > manifest codec: mp4a.40.2,ADTS data:type:2,sampleingIndex:4[44100Hz],channelConfig:2
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > parsed codec:mp4a.40.5,rate:44100,nb channel:2
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > audio mp4 timescale :44100
logger.js:39 [log] > creating sourceBuffer(audio/mp4;codecs= mp4a.40.2)
logger.js:39 [log] > creating sourceBuffer(video/mp4;codecs=avc1.4d001e)
logger.js:39 [log] > main track:audio,container:audio/mp4,codecs[level/parsed]=[ mp4a.40.2/mp4a.40.5]
logger.js:39 [log] > main track:video,container:video/mp4,codecs[level/parsed]=[avc1.4d001e/avc1.4d4015]
logger.js:39 [log] > Parsed audio,PTS:[0.000,11.285],DTS:[0.000/11.285],nb:486,dropped:0
logger.js:39 [log] > Parsed video,PTS:[0.066,11.343],DTS:[0.000/11.277],nb:338,dropped:0
logger.js:39 [log] > engine state transition from PARSING to PARSED
logger.js:39 [log] > latency/loading/parsing/append/kbps:55/228/178/0/13825
logger.js:39 [log] > main buffered : [0.000,11.277]
logger.js:39 [log] > engine state transition from PARSED to IDLE
logger.js:39 [log] > switching to level 5
logger.js:39 [log] > loading playlist for level 5
logger.js:39 [log] > engine state transition from IDLE to WAITING_LEVEL
logger.js:39 [log] > target start position:0
logger.js:39 [log] > target start position not buffered, seek to buffered.start(0) 0.000488
logger.js:39 [log] > adjust currentTime from 0 to 0.000488
logger.js:39 [log] > media seeking to 0.000
logger.js:39 [log] > media seeked to 0.000
logger.js:39 [log] > level 5 loaded [1,19],duration:187.82
logger.js:39 [log] > engine state transition from WAITING_LEVEL to IDLE
logger.js:39 [log] > Loading 2 of [1 ,19],level 5, currentTime:0.021,bufferEnd:11.277
logger.js:39 [log] > engine state transition from IDLE to FRAG_LOADING
logger.js:39 [log] > Loaded  2 of [1 ,19],level 5
logger.js:39 [log] > engine state transition from FRAG_LOADING to PARSING
logger.js:39 [log] > Parsing 2 of [1 ,19],level 5, cc 0
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > level switch detected
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > manifest codec: mp4a.40.2,ADTS data:type:2,sampleingIndex:4[44100Hz],channelConfig:2
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > parsed codec:mp4a.40.5,rate:44100,nb channel:2
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [log] > audio mp4 timescale :44100
logger.js:39 [log] > main track:audio,container:audio/mp4,codecs[level/parsed]=[ mp4a.40.2/mp4a.40.5]
logger.js:39 [log] > main track:video,container:video/mp4,codecs[level/parsed]=[avc1.4d001e/avc1.4d401f]
blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902 [warn] > **Injecting 57 audio frame @ 10s due to 1331 ms gap.**(anonymous function) @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3902remuxAudio @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3460remux @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:3061remux @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:1587push @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:1563push @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:682(anonymous function) @ blob:http://dailymotion.github.io/5102cd89-01b6-4ce5-ab0d-eaedc83ca441:758
logger.js:39 [log] > Parsed audio,PTS:[10.000,21.331],DTS:[10.000/21.331],nb:488,dropped:0
logger.js:39 [log] > Parsed video,PTS:[11.377,21.387],DTS:[11.311/21.321],nb:300,dropped:0
logger.js:39 [log] > engine state transition from PARSING to PARSED
logger.js:39 [log] > latency/loading/parsing/append/kbps:59/317/102/17/67368
logger.js:39 [log] > main buffered : [0.000,21.320]
logger.js:39 [log] > engine state transition from PARSED to IDLE
logger.js:39 [log] > Loading 3 of [1 ,19],level 5, currentTime:0.459,bufferEnd:21.320
logger.js:39 [log] > engine state transition from IDLE to FRAG_LOADING
logger.js:39 [log] > Loaded  3 of [1 ,19],level 5
logger.js:39 [log] > engine state transition from FRAG_LOADING to PARSING
(more)
mangui commented 7 years ago

Hi @frankpaolino plz recheck against latest hls.js/master or demo page, issue should be fixed

frankpaolino commented 7 years ago

@mangui Thank you, it does indeed look fixed!