Open talaysa opened 3 years ago
Support for playing subtitles on Chromecast is a feature that I would like, as well. I spent some time yesterday exploring this issue, and will share my findings.
Ideally, I'd like this to trigger some discussion because there are a few moving parts and proper support for this feature would require touching both clappr-core
and hlsjs-playback
.
For my next step, I believe that I can write a small plugin to extend (ie: monkey-patch) this clappr-chromecast-plugin to add support for settings.playback.externalTracks
. Even if successful, it still won't be able to support in-stream text tracks.
before digging into code, lets start with some Clappr settings to test text tracks:
// -----------------------------------------------
// references:
// https://github.com/clappr/clappr/issues/1477
// -----------------------------------------------
var initialize_text_tracks = function() {
var video = document.querySelector('video')
if (!video) return
var textTracks = video.textTracks
if (!textTracks || !textTracks.length) return
for (var i=0; i < textTracks.length; i++) {
if (textTracks[i].mode === 'showing') return
}
// turn on the first subtitles text track (which is always "Disabled") to display the "CC" icon/menu in the media-control panel
textTracks[0].mode = 'showing'
}
// -----------------------------------------------
// references:
// http://demo.theoplayer.com/closed-captions-subtitles
// https://github.com/videojs/video.js/tree/v7.10.2/docs/examples/elephantsdream
// -----------------------------------------------
var player = new Clappr.Player({
source: 'https://cdn.theoplayer.com/video/elephants-dream/playlist-single-audio.m3u8', // in-stream WebVTT: Chinese, French
poster: 'https://demo.theoplayer.com/hubfs/Demo_zone/elephants-dream.jpg',
height: 360,
width: 640,
plugins: [ChromecastPlugin],
chromecast: {
appId: '9DFB77C0',
media: {
type: ChromecastPlugin.Movie,
title: 'Elephants Dream',
subtitle: '2006 Dutch computer animated science fiction fantasy experimental short film produced by Blender Foundation'
}
},
playback: {
crossOrigin: 'anonymous',
externalTracks: [{
lang: 'en-US',
label: 'English',
kind: 'subtitles',
src: 'https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.en.vtt'
},{
lang: 'sv',
label: 'Swedish',
kind: 'subtitles',
src: 'https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.sv.vtt'
},{
lang: 'ru',
label: 'Russian',
kind: 'subtitles',
src: 'https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.ru.vtt'
},{
lang: 'ja',
label: 'Japanese',
kind: 'subtitles',
src: 'https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.ja.vtt'
},{
lang: 'ar',
label: 'Arabic',
kind: 'subtitles',
src: 'https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.ar.vtt'
}]
},
events: {
onPlay: initialize_text_tracks
}
});
observations without any changes to Clappr:
necessary changes to settings to continue with testing:
var player = new Clappr.Player({
source: 'https://d2zihajmogu5jn.cloudfront.net/elephantsdream/ed_hd.mp4',
...
});
the "Add Advanced Features" section in the Chromecast docs for building a Chrome sender app outlines how to manage (external) text tracks
as a first step, I confirmed that the above methodology will work with the Clappr Chromecast receiver app by making the following changes to chromecast.js and testing them while running the webpack dev server:
get activeTrackIds() {
let trackId = this.container.closedCaptionsTrackId
return (trackId >= 0)
? [trackId]
: []
}
loadMedia() {
this.container.pause()
let src = this.container.options.src
Log.debug(this.name, 'loading... ' + src)
let mediaInfo = this.createMediaInfo(src)
let request = new chrome.cast.media.LoadRequest(mediaInfo)
request.autoplay = true
request.activeTrackIds = this.activeTrackIds
if (this.currentTime) {
request.currentTime = this.currentTime
}
this.session.loadMedia(request, (mediaSession) => this.loadMediaSuccess('loadMedia', mediaSession), (e) => this.loadMediaError(e))
}
createMediaInfo(src) {
let mimeType = ChromecastPlugin.mimeTypeFor(src)
let mediaInfo = new chrome.cast.media.MediaInfo(src)
mediaInfo.contentType = this.options.contentType || mimeType
mediaInfo.customData = this.options.customData
let metadata = this.createMediaMetadata()
mediaInfo.metadata = metadata
let tracks = this.createMediaTracks()
mediaInfo.tracks = tracks
return mediaInfo
}
// ---------------------------------------------
// references:
// https://developers.google.com/cast/docs/chrome_sender/advanced
// ---------------------------------------------
createMediaTracks() {
const tracks = []
const langs = [['en','English'],['sv','Swedish'],['ru','Russian'],['ja','Japanese'],['ar','Arabic']]
for (let i=0; i < langs.length; i++) {
const [language, name] = langs[i]
const url = `https://raw.githubusercontent.com/videojs/video.js/v7.10.2/docs/examples/elephantsdream/captions.${language}.vtt`
const track = new chrome.cast.media.Track(i, chrome.cast.media.TrackType.TEXT)
track.trackContentId = url
track.trackContentType = 'text/vtt'
track.subtype = chrome.cast.media.TextTrackType.SUBTITLES
track.language = language
track.name = name
track.customData = null
tracks.push(track)
}
return tracks
}
having confirmed this will work, the next step was to reimplement createMediaTracks
:
createMediaTracks() {
const textTracks = this.container.closedCaptionsTracks // [{id, name, track}]
return textTracks.map(textTrack => {
const track = new chrome.cast.media.Track(textTrack.id, chrome.cast.media.TrackType.TEXT)
//track.trackContentId = undefined
//track.trackContentType = undefined
track.subtype = chrome.cast.media.TextTrackType.SUBTITLES
track.language = textTrack.track.language
track.name = textTrack.name
track.customData = null
return track
})
}
this quickly hit a roadblock
textTracks
is obtained from the getter: closedCaptionsTracks
textTracks[0].track === document.querySelector('video').textTracks[0]
<track>
elementthoughts..
settings.playback.externalTracks
and DOM <track>
elementsexternal text tracks:
this.container.closedCaptionsTrackId
to the correct URL
this.container.closedCaptionsTrackId
refers to the complete list of this.container.closedCaptionsTracks
id
attribute that can be used to query the underlying <track>
element, if any
clappr-core
creates DOM <track>
element, it doesn't add any (unique) id
attributesettings.playback.externalTracks
are always loaded BEFORE in-stream text tracksin-stream HLS text tracks:
hlsjs-playback
depends upon hls.js
hls.js
API fires several interesting eventshlsjs-playback
adds a listener, but doesn't inspect the data for its underlying URLsI think that's a fairly thorough summary of where I'm at right now.
As I said, I'm going after the low-hanging fruit.. and will report back with any progress.
imho, it works great; feedback (and PRs) welcome :smiley:
PS, the live demo doesn't set crossOrigin
, because that would break video playback. The result is that the text tracks won't play in the web browser, but they'll play fine on Chromecast.
PS, regarding clappr#1477, the initialize_text_tracks
workaround included in the example Clappr settings (above) is no-longer needed when using this enhanced plugin; it performs this task automagically (and more cleanly).
Hi, Is there any subtitle support on chromecast? We have external track with VTT format, how we can send them with chromecast?
Thanks, Salih