search

shaka-project / shaka-packager

A media packaging and development framework for VOD and Live DASH and HLS applications, supporting Common Encryption for Widevine and other DRM Systems.
https://shaka-project.github.io/shaka-packager/
Other
1.97k stars 507 forks source link

WebVTT: VTT cue incorrect segmentation and HLS subtitle playlist incorrect #1135

Closed vish91 closed 5 months ago

vish91 commented 1 year ago

System info

Operating System: macOS Shaka Packager Version: 2.6.1 or newer

Issue and steps to reproduce the problem

When generating an HLS package, if a VTT subtitle is passed as input. If that VTT does not have any text cues for initial few minutes the VTT playlist generated is segmented incorrectly and the playlist manifest indexes are incorrect. e.g media is 40 minutes and if the VTT cues start at 20 mins or 30 mins into the media. The VTT playlist generated looks like :

Input:

$ cat es-subtitles-1.vtt
WEBVTT

00:38:14.837 --> 00:38:17.881 align:middle
No lo sé. ¡No lo sé!

00:38:19.550 --> 00:38:21.301 align:middle
Arréstalo, Danno.

00:38:21.635 --> 00:38:23.387 align:middle
Vamos.

Output:

$ cat /tmp/shaka/1131401_fp_hls/es-subtitles-1/stream_vtt.m3u8
#EXTM3U
#EXT-X-VERSION:6
## Generated with https://github.com/google/shaka-packager version v2.6.1-634af65-release
#EXT-X-TARGETDURATION:7
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:6.000,
seg_1.vtt
#EXTINF:6.000,
seg_2.vtt
#EXTINF:6.000,
seg_3.vtt
#EXTINF:6.000,
....
...
$ cat /tmp/shaka/1131401_fp_hls/es-subtitles-1/seg_1.vtt
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:9000

00:38:14.837 --> 00:38:17.881 align:center
No lo sé. ¡No lo sé!

Packager Command: packager 'in=CBS_NCIS_366_AA_6CH_30M_2_1778839_3_aac_128.mp4,stream=audio,segment_template=/tmp/shaka/1131401_fp_hls/CBS_NCIS_366_AA_6CH_30M_2_1778839_3_aac128/seg$Number$.ts,lang=en,playlist_name=CBS_NCIS_366_AA_6CH_30M_2_1778839_3_aac_128/stream.m3u8,hls_group_id=audio_aac,hls_name=English,drm_label=audio' 'in=CBS_NCIS_366_AA_6CH_30M_2_1778839_375.mp4,stream=video,segment_template=/tmp/shaka/1131401_fp_hls/CBS_NCIS_366_AA_6CH_30M_2_1778839375/seg$Number$.ts,iframe_playlist_name=CBS_NCIS_366_AA_6CH_30M_2_1778839_375/stream_iframe.m3u8,playlist_name=CBS_NCIS_366_AA_6CH_30M_2_1778839_375/stream.m3u8,drm_label=sd' 'in=CBS_NCIS_366_AA_6CH_30M_2_1778839_1500.mp4,stream=video,segment_template=/tmp/shaka/1131401_fp_hls/CBS_NCIS_366_AA_6CH_30M_2_17788391500/seg$Number$.ts,iframe_playlist_name=CBS_NCIS_366_AA_6CH_30M_2_1778839_1500/stream_iframe.m3u8,playlist_name=CBS_NCIS_366_AA_6CH_30M_2_1778839_1500/stream.m3u8,drm_label=sd' 'in=es-subtitles-1.vtt,stream=text,segment_template=/tmp/shaka/1131401_fphls/es-subtitles-1/seg$Number$.vtt,playlist_name=es-subtitles-1/stream_vtt.m3u8,hls_group_id=cbsi_webvtt,language=es-MX,hls_name=Mexican Spanish' --clear_lead 0 --segment_duration 6 --hls_master_playlist_output /tmp/shaka/1131401_fp_hls/master.m3u8 --enable_raw_key_encryption --keys label=audio:key_id=K1:key=K2,label=sd:key_id=K3:key=K4 --iv IV1 --protection_scheme cbcs --pssh --hls_key_uri ''

What is the expected result? Expected result would be VTT playlist generated with the VTT Cue text in the correct segment matching the timestamp of the vtt cue. In this example: first VTT cue is at 38 min and 14 sec , so the that cue should be in 383rd segment since I am making 6 second segments here.

What happens instead? Instead as we see in example above the cue shows up in seg_1.vtt

<Please attach the input files or email to shaka-packager-issues@google.com.>

vish91 commented 1 year ago

@joeyparrish @kqyang I created this issue but found this in older issues https://github.com/shaka-project/shaka-packager/issues/697

Could you share what was the reason for 10 minute mark limit ? Was it some optimization and can we remove it ? use case:

  1. Even if subtitles are at 15-20 min mark , packager currently is packaging them incorrectly to show up at wrong segment. Its an edge case but we can have some content where subtitles start past 10 min mark.
  2. I am trying to build support for Forced Narrative subtitles (i have a PR open for that ) . Forced Narrative subtitles can be at any timestamp in the content and may not be within this 10 minute condition , so for that as well we need to remove this condition.

Happy to submit a PR just needed to discuss and get more details on what and how this is implemented.

vish91 commented 1 year ago

@joeyparrish any thoughts ?

cosmin commented 5 months ago

Let's merge this discussion into #697

vish91 commented 5 months ago

@cosmin we can close this out . we made a PR for this feature to make this bias optional and its merged in as well. https://github.com/shaka-project/shaka-packager/pull/1330