Support external vtt files when playing mp4 files

[rosenbjerg]

Shaka supports playing a regular mp4 file, but does not recognize track elements inside the video container nor provide any other method to add external .vtt files as subtitle/caption tracks.

--

I would like to be able to add external subtitle files, in WebVTT format of course, so they are available for selection through the captions menu in the UI. I think it would make the most sense to support parsing the track elements inside the video element that is transformed by Shaka. Otherwise, a method on the player or ui instances, to add one or more text tracks, would also be nice.

--

I have considered the possibility of creating a custom manifest parser and build a simple manifest format specifying the path to the mp4 file and a number of vtt files. I think that seems like a quite impractical solution, and I've not yet finished researching whether it would be possible to do that way.

[joeyparrish]

In the general case, you can't create a manifest parser to point to a flat MP4 file. Playing DASH, HLS, or other ABR content formats goes through MediaSource APIs, which requires fragmented formats (fMP4/CMAF, for example). A plain, flat MP4 is playing using the video's src attribute, not MediaSource, so that we can play any MP4 the browser could play, not just fMP4.

So I think it's not worth your time to try creating a custom manifest parser for MP4s.

As for external VTT files, when we are streaming DASH/HLS, we can take the external VTT and side-load it into the parsed manifest. The internal component StreamingEngine is responsible for fetching text as well as video & audio. It then sends the text through TextEngine to be parsed and sent to the display component.

None of these components (StreamingEngine, TextEngine, etc.) are in use when we play in src mode, as we do for plain MP4s.

So side-loaded text w/ plain MP4s is not something we can support right now.

But the good news is that you can just skip Shaka Player completely and use plain HTML5 video for this. For example (adapted from an MDN article on the subject):

<video id="video" controls src="video/sintel-short.mp4">
   <track label="English" kind="subtitles" srclang="en" src="captions/vtt/sintel-en.vtt" default>
   <track label="Deutsch" kind="subtitles" srclang="de" src="captions/vtt/sintel-de.vtt">
   <track label="Español" kind="subtitles" srclang="es" src="captions/vtt/sintel-es.vtt">
</video>

In my opinion, for cases like this (VTT + plain MP4), there's really no need for the complexity of Shaka Player. The browser already has everything you need built in.

Does this help?

[rosenbjerg]

Okay, thank you for saving me the time of trying to create a manifest parser just to realize it wont work.

I know that the browser supports the kind of playback that I'm after, but I don't know how to support casting to chromecast using a video player. It seemed to me that shaka-player would be my best bet for supporting chromecasting. But I do see that this isn't what shaka is designed for. Do you know if it is possible to support "optimized" casting to chromecast through a regular video player? Not that I've gotten that for with implementing shaka into the project yet.

[rosenbjerg]

And thank you for the surprisingly quick response!

[joeyparrish]

Chrome's native video controls (<video> with the controls attribute) should have a built-in cast button which will work for plain MP4s.

[rosenbjerg]

Okay, I will look more into that. I did not know that was supported, as I haven't seen that anywhere yet. Thanks for the help! I will close this issue/question :)