audio frames are getting skipped while playback

yash1994 commented 10 months ago

What version of Hls.js are you using?

1.4.12

What browser (including version) are you using?

Brave Version 1.61.109 Chromium: 120.0.6099.144 (Official Build) (64-bit)

What OS (including version) are you using?

Ubuntu 22.04.2 LTS

Test stream

https://video.gumlet.io/5f462c1561cf8a766464ffc4/659534fac81fcf3d8d364619/main.m3u8

Configuration

{
    backBufferLength: 30000
}

Additional player setup steps

No response

Checklist

[X] The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
[X] The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
[X] The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
[X] The stream has correct Access-Control-Allow-Origin headers (CORS)
[X] There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

Play the following HLS URL with HLS.js https://video.gumlet.io/5f462c1561cf8a766464ffc4/659534fac81fcf3d8d364619/main.m3u8

Expected behaviour

Audio stream playback should be the same as the audio stream in the original mp4 video file.

What actually happened?

During the first 4-6 seconds of the video, a few audio frames are getting dropped. The person in the video (original) speaks the following sentence: "Alright, if you are watching this video, you are more than likely a local business owner..."

When played with HLS.js, the "business" word is dropped and the following sentence is audible: "Alright, if you are watching this video, you are more than likely a local owner..."

Checking on underlying fMP4 audio files plays perfectly fine (in sync with original audio) as fMP4 format.

Console output

Complete Debug Log file: https://gumlet.sgp1.digitaloceanspaces.com/video/hls_js_debug_log.txt

Chrome media internals output

No response

robwalch commented 10 months ago

It looks like the the video track has a start time ~0.26s later than the audio which starts at 0. HLS.js is aligning the start of the program to the video track. The first audio segment is appended at 0 (negative starts are not supported), but then to keep audio and video in sync, the remaining segments are appended according with the same same offset (- video start) as all the other media (second audio segment starts @ 3.75s) which overlaps and crops the end of the first segment.

robwalch commented 10 months ago

For fragmented mp4 content we should align on the earliest start time of the two tracks. This would lead to start gap in the video track, but prevent overlapping appends of audio. I don't have a time table for any such fix.

video-dev / hls.js