search

ossrs / oryx

Oryx(SRS Stack) is an all-in-one, out-of-the-box, and open-source video solution for creating online video services, including live streaming and WebRTC, on the cloud or through self-hosting.
https://ossrs.io/oryx
MIT License
519 stars 111 forks source link

Support output HLS with WebVTT for transcript #168

Closed winlinvip closed 6 months ago

winlinvip commented 6 months ago

Specification: https://www.w3.org/TR/webvtt1/

Hello WebVTT

Example:

<html>
 <head>
  <title>WebVTT</title>
 </head>
 <body>
  <video controls autoplay src="avatar.mp4">
   <track default src="track.vtt">
  </video>
 </body>
</html>
WEBVTT

00:05.000 --> 00:09.000
<v Roger Bingham>We are in New York City

00:10.000 --> 00:16.000
<v Roger Bingham>We’re actually at the Lucern Hotel, just down the street

00:16.000 --> 00:18.000
<v Roger Bingham>from the American Museum of Natural History

00:18.000 --> 00:20.000
<v Roger Bingham>And with me is Neil deGrasse Tyson

00:20.000 --> 00:22.000
<v Roger Bingham>Astrophysicist, Director of the Hayden Planetarium

00:22.000 --> 00:24.000
<v Roger Bingham>at the AMNH.

00:24.000 --> 00:26.000
<v Roger Bingham>Thank you for walking down here.

Access the HTML page in web server:

Image

WebVTT for HLS

Convert mp4 to HLS:

ffmpeg -t 30s -i avatar.mp4 -c:v h264 -profile:v baseline -r 25 -g 25 -c:a aac -hls_time 10 -y output.m3u8

The m3u8 file is something like bellow:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.520000,
output0.ts
#EXTINF:9.480000,
output1.ts
#EXTINF:10.040000,
output2.ts
#EXT-X-ENDLIST

The first WebVTT segment file, subtitle0.vtt:

WEBVTT

00:05.000 --> 00:09.000
<v Roger Bingham>We are in New York City

The second WebVTT segment file, subtitle1.vtt:

WEBVTT

00:10.000 --> 00:16.000
<v Roger Bingham>We’re actually at the Lucern Hotel, just down the street

00:16.000 --> 00:18.000
<v Roger Bingham>from the American Museum of Natural History

00:18.000 --> 00:20.000
<v Roger Bingham>And with me is Neil deGrasse Tyson

The last WebVTT segment file, subtitle2.vtt:

WEBVTT

00:20.000 --> 00:22.000
<v Roger Bingham>Astrophysicist, Director of the Hayden Planetarium

00:22.000 --> 00:24.000
<v Roger Bingham>at the AMNH.

00:24.000 --> 00:26.000
<v Roger Bingham>Thank you for walking down here.

Create a subtitles.m3u8 for WebVTT:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.520000,
subtitle0.vtt
#EXTINF:9.480000,
subtitle1.vtt
#EXTINF:10.040000,
subtitle2.vtt
#EXT-X-ENDLIST

And need a index.m3u8 to associate video and subtitle:

#EXTM3U
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,FORCED=NO,URI="subtitles.m3u8",LANGUAGE="en"
#EXT-X-STREAM-INF:BANDWIDTH=1118592,CODECS="mp4a.40.2, avc1.64001f",RESOLUTION=640x360,SUBTITLES="subs"
output.m3u8

Access the index.m3u8 page in Safari browser:

image

HLS Live Stream with WebVTT

Safari:

image

VLC:

image

SRS Player(hls.js):

image

winlinvip commented 6 months ago

Fixed in v5.14.2 13f0a8f