search

video-dev / hls.js

HLS.js is a JavaScript library that plays HLS in browsers with support for MSE.
https://hlsjs.video-dev.org/demo
Other
14.42k stars 2.55k forks source link

DIFFERENCE IN FRAME #6490

Open ravinair24 opened 3 weeks ago

ravinair24 commented 3 weeks ago

What version of Hls.js are you using?

1.4.0

What browser (including version) are you using?

Chrome

What OS (including version) are you using?

Linux

Test stream

https://streams.abs.tv:8443/videos/82/Videos/RTV_PRG_Whickers_World_S03E06.mp4/chunk.m3u8

Configuration

{autoStartLoad: true,
startPosition : -1,
debug: true,
maxBufferLength: 30,
maxMaxBufferLength: 90,
maxBufferSize: 60*1000*1000,
maxBufferHole: 0,}

Additional player setup steps

No response

Checklist

Steps to reproduce

  1. An additional 2-3 frames get automatically added at the beginning of video which necessarily means that the video Timecodes with seek is off by 2-3 frames

Expected behaviour

should not add additional frames in the beginning or the end

What actually happened?

An additional 2-3 frames get automatically added at the beginning of video which necessarily means that the video Timecodes with seek is off by 2-3 frames

Console output

[log] > [stream-controller]: Media seeked to 0.080
hls.js?r=11062024111906:8752 [log] > [stream-controller]: media seeking to 0.040, state: IDLE
hls.js?r=11062024111906:8752 [log] > [audio-stream-controller]: media seeking to 0.040, state: STOPPED
hls.js?r=11062024111906:8752 [log] > [subtitle-stream-controller]: media seeking to 0.040, state: IDLE
2VM1755:2515 Uncaught

Chrome media internals output

No response

robwalch commented 3 weeks ago

HLS.js appends audio and video in MSE SourceBuffers based on the sample timestamps found in the media. Appended media is decoded and rendered by the browser according to its mapping of decode and presentation timestamps (DTS/PTS) to HTMLMediaElement currentTime. Some common causes for video frames not aligning with the original source are:

  1. Audio priming delay: AAC encoding adds samples to the start that cannot be clipped in TS or MSE which offsets video
  2. Composition time (CTS: the difference between PTS and DTS) applied to DTS/PTS on HTMLMediaElement time. Most MSE implementations add CTS the position of appended segments.
robwalch commented 3 weeks ago

Probing your first segment shows that audio starts 0.12s before video:

ffprobe -v error -hide_banner -select_streams v -show_frames -i 'https://streams.abs.tv:8443/videos/82/Videos/RTV_PRG_Whickers_World_S03E06.mp4/n_0_0_0.ts'

[FRAME]
media_type=audio
stream_index=1
key_frame=1
pts=0
pts_time=0.000000
pkt_dts=0

[FRAME]
media_type=video
stream_index=0
key_frame=1
pts=10800
pts_time=0.120000
pkt_dts=3600
pkt_dts_time=0.040000
ravinair24 commented 3 weeks ago

Thanks a bunch Rob for your prompt response.

The probe is correct, the video does start at frame 3. However for some reason HLS.js is adding 2-3 frames at the beginning, which means I can see the video only at frame 6. Although, when I use videojs' native HLS, the video starts exactly at frame

  1. I like HLS.js as it gives me a lot of flexibility, however frame accuracy is sacrosanct for the app that I am building. Not sure if you can do something about this. I would be very happy if something can be done.

Regards,

RAVI NAIR

On Thu, 13 Jun 2024 at 21:42, Rob Walch @.***> wrote:

HLS.js appends audio and video in MSE SourceBuffers based on the sample timestamps found in the media. These then decoded and rendered by the browser according to its mapping of decode and presentation timestamps (DTS/PTS) to HTMLMediaElement currentTime. Some common causes for video frames not aligning with the original source are:

  1. Audio priming delay: AAC encoding adds samples to the start that cannot be clipped in TS or MSE which offsets video
  2. Composition time (lets call this CTS - it is the difference between PTS and DTS) applied to DTS/PTS on HTMLMediaElement time (CTS is added to the position of appended segments)

— Reply to this email directly, view it on GitHub https://github.com/video-dev/hls.js/issues/6490#issuecomment-2166126295, or unsubscribe https://github.com/notifications/unsubscribe-auth/AT4N52BZ6Z2ICAK27VZLJ6LZHHAHBAVCNFSM6AAAAABJELYUBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRWGEZDMMRZGU . You are receiving this because you authored the thread.Message ID: @.***>