Captions with streaming libraries

friday commented 6 years ago

I'm collecting all the info I have in one issue (and intend to keep it updated until we can support multiple tracks of the same language and the streaming demos in the main documentation have been updated with captions).

Plyr 3.3.10 added a property captions.update for cases when you want Plyr to update the language selection if tracks are added after the instance is initialized.

Adding a track dynamically:

new Plyr(video, {captions: {update: true}});
const track = video.addTextTrack('subtitles', 'Volapük', 'vo');
track.addCue(new VTTCue(0, 4, 'Ven lärnoy püki votik, vödastok plösenon fikulis.'));
track.addCue(new VTTCue(5, 9, 'Mutoy ai dönu sukön vödis nesevädik, e seko nited paperon.'));

Or via the dom:

new Plyr(video, {captions: {update: true}});
const track = document.createElement('track');
Object.assign(track, {
  label: 'English',
  srclang: 'en',
  default: true,
  src: 'https://cdn.plyr.io/static/demo/View_From_A_Blue_Moon_Trailer-HD.en.vtt'
});
video.appendChild(track);

captions.update is false by default, since the tracks created with it enabled will likely not have any cues, and hence selecting it won't "work". You can however make it work by listening to the languagechange event and handling the necessary update there.

Dash.js Status: Seems to works perfectly (demo), and we don't even have to handle the languagechange event. 🍾 🍰

Hls.js Status: Experimental (demo with multi-track VTT, demo with single-track embedded captions). It's mostly working since v0.10, but seems to require a timeout-hack (which is unsafe). Preloading all tracks would probably solve this, but they don't yet support this. 0.10 also has some other issues that has yet to be fixed.

Shaka Player Shaka Player only creates one TextTrack withouts any other useful information at all. So it's not possibly to use it with Plyr currently, at least if you want to be able to switch languages. You can however add your own html5 <track>-elements if your captions are in the VTT format. You also should keep captions.update disabled (default) since otherwise it will just create the option "Shaka Player TextTrack".

Tasklist:

[x] Keep internal reference to track index rather than just language code (which may not be unique and hence impossible to differentiate in the languagechange event). See #1017 and PR #1015).
[x] Update docs with Plyr 3.3.10 (or newer) and these examples.
[ ] Document support in the wiki or main documentation.
[ ] Expose the internal method to set languages for Shaka Player (I think we should wait with this until we have Plugin support and do it that way instead). See #750.

Mr-Black-Dahlia commented 6 years ago

Amazing work! For some reason on my end I can see the different caption options for HLS but they are not showing themselves on the player. Dash is working perfectly though!

friday commented 6 years ago

Yeah. Have seen that in HLS too occasionally. Like the German track in the demo ^. Not sure what's happening there.

friday commented 6 years ago

Updated the hls.js demo to use the hls.js canary version (https://cdn.jsdelivr.net/npm/hls.js@canary) since that seems to fix all the issues I had in the demo (the German captions not working and some captions not showing the first time).

gurupras commented 6 years ago

Is there any way we can we get to work in the general case? I'd like to dynamically add textTracks to HTML5 videos. How can I achieve this? Here's an example

friday commented 6 years ago

This should work:

new Plyr(video, {captions: {update: true}});
const track = video.addTextTrack('subtitles', 'Volapük', 'vo');
track.addCue(new VTTCue(0, 3, 'Your first cue'));

Not sure why you're doing it via the dom. It should work too, but you'll need more code to achieve this. I haven't tried for this issue, but I'm pretty sure I have done this before and it created the textTracks objects. Both plyr.media.textTracks.length and plyr.media.children.length is 0 in your demo, so something isn't working.

gurupras commented 6 years ago

Not sure why you're doing it via the dom.

I was checking if my code works for the native HMTL5 player (it does) and was hence doing it via the DOM. Why is it that I'm able to add a TextTrack and have it work via the native player but not through Plyr?

In my usecase, ideally, I'd like to set it up to use src rather than go through each cue and add it manually.

friday commented 6 years ago

I see. If you have URLs rather than creating them as blobs I can see the benefit to using the dom. Either way. make sure you actually get this working without Ply first, and that video.textTracks.length and video.children.length isn't 0. Otherwise it has nothing to do with Plyr.

gurupras commented 6 years ago

It does work without Plyr: Here's the same example with the native player.

friday commented 6 years ago

Neither the video nor the caption loads. You have probably installed a browser plugin like "Allow-Control-Allow-Origin: *" or "Cors toggle" to override cors.

Also, you need to enable the textTrack:

document.querySelector('#video').textTracks[0].mode = 'showing'; // use 'hidden' instead of 'showing' for Plyr

Not sure how to do that with the dom, but probably the default property.

gurupras commented 6 years ago

right, sorry, I do have that enabled. I will try and host a video and allow CORS to get this working

On Thu, Jun 7, 2018 at 3:14 PM, friday notifications@github.com wrote:

Neither the video nor the caption loads. You have probably installed a browser plugin like "Allow-Control-Allow-Origin: *" or "Cors toggle" install to override cors.

Also, you need to enable the textTrack:

document.querySelector('#video').textTracks[0].mode = 'showing'; // use 'hidden' instead of 'showing' for Plyr

Not sure how to do that with the dom, but probably the default property.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sampotts/plyr/issues/994#issuecomment-395533300, or mute the thread https://github.com/notifications/unsubscribe-auth/ACzB11c9LryMU2Fvrk58eK5IQ4xMRdu7ks5t6XuFgaJpZM4UVu22 .

friday commented 6 years ago

I forked your pen, changed the sources, set default to true, and used srcLang (not language). It works: https://codepen.io/fullkornslimpa/pen/ZRBYrR?editors=1011

gurupras commented 6 years ago

Perfect! Thanks for taking the time out to resolve this.

On Thu, Jun 7, 2018 at 3:54 PM, friday notifications@github.com wrote:

I forked your pen, changed the sources, set default to true, and used srcLang (not language). It works: https://codepen.io/ fullkornslimpa/pen/ZRBYrR?editors=1011

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sampotts/plyr/issues/994#issuecomment-395544392, or mute the thread https://github.com/notifications/unsubscribe-auth/ACzB1y8lyyzA7K8ExA78qooJN3a4Etgtks5t6YTkgaJpZM4UVu22 .

friday commented 6 years ago

Plyr 3.3.12 was just released, with support for currentTrack getter/setter for the track index, in addition to still supporting the previous language getter/setter. I think that's about what we can do from our side for support with hls.js and dash.js. Shaka Player needs more, but it's outside of the scope of what I think we need to support/handle, and not a priority for me since they don't support changing tracks via the native HTML5 video controls and/or proper textTracks (#750). This may come to change if I need to switch from hls.js to Shaka Player myself, since the latter seems more stable.

In addition to this, I noticed dash.js didn't need to handle the languagechange event at all. I just assumed it did, coming from hls.js.

The Plyr documentation has been updated with the same codepen links as in here.

We could still use some kind of compatibility table in the wiki or main docs I think. Different providers or libraries support for captions, handling quality and other features.

sampotts / plyr

Captions with streaming libraries #994