Add support for segmented WebVTT subtitles

[sandersaares]

Currently, dash.js only appears to support TTML in segmented form. It would be desirable for us if it also supported segmented WebVTT.

Example content of this type is available at https://github.com/Axinom/dash-test-vectors

[dsparacio]

Thank you for the issue report. Agree this should be supported and the content is a big help. Not sure if we can get it into 2.0 but a great feature addition for the next release.

[dsparacio]

With MPD http://media.axprod.net/TestVectors/v6-Clear/Manifest_1080p.mpd mimeType="application/mp4" codecs="wvtt"

Currently in dash.js the expected mimeType is "text/vtt". Can you try with this and update on your findings if possible?

[sandersaares]

I observe no subtitle playback taking place if I change the mimeType to text/vtt, as is expected - from what I see of the code there is just a plain-text WebVTT parser in there but embedded WebVTT requires ISO 14496-30 to be implemented. Granted, perhaps I simply miss the right code to look at since I am not very experienced with the codebase.

[TobbeMobiTV]

Yes, we need to add the corresponding mechanism that we have for TTML segments. However, as is evident from ISO 14496-30, the handling of timing is quite different in WebVTT segments from TTML segments, so it is not trivial. The main entry point for adding this functionality is in TextSourceBuffer.js below line 177 dealing with fragmentedText. We need to add another parser here which extracts the timing and text from WebVTT segments and create captions.

[TobbeMobiTV]

With small changes to DashManifestExtensions.js and SourceBufferExtensions.js I get all the languages to appear and the WebVTT segments to be downloaded with the same mechanism as TTML segments. I can switch between them. Remains to get the extraction of the WebVTT segment data working in TextSourceBuffer.js. Will hopefully get some time to look at it later this week.

[TobbeMobiTV]

I start to get this working in basic mode. Since WebVTT according to 14496-30 is sent as new iso boxes inside the samples, I needed to extend the iso_boxer.min.js library with parsers for vtte (empty), vttc(container), payl(payload). I haven't done other WebVTT boxes. Testing with http://media.axprod.net/TestVectors/v6-Clear/Manifest_1080p.mpd the first track (lang="ch") is strange since there are empty vttc boxes starting in segment 0006.m4s. That makes the content somewhat hard to use for testing since the default track is the first one.

I'm not ready for PR yet. Two questions to answer is how we should extend the iso_boxer.js file and how complete this implementation of WebVTT should be?

For the first, I think we should contribute the additional boxes to https://github.com/madebyhiro/codem-isoboxer. We may also choose to include iso_boxer.js as a non-minified file in our external source, to enable debugging.

[sandersaares]

Looks like the ch track is defective, indeed. The root cause appears to be a defect in our subtitle processing workflow. I will try to get that track updated with fixed subtitles.

[sandersaares]

The ch track on v6-Clear is now fixed. Looks like our other test videos have the same defect with the ch track, those will also be fixed today.

[TobbeMobiTV]

@sandersaares Yes, I now get subtitles for lang='ch'. According to https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes the language is Chamorro, but the characters look more Chinese to me.

[sandersaares]

Indeed - looks like a bad language code. But I will ignore that for now since it's a cosmetic issue.

[sandersaares]

Okay okay it upset the perfectionist in me, so the language code is now also fixed, both in the manifests and the boxes of the initialization segments. The new language code for that adaptation set is zh-Hant-CN.

[TobbeMobiTV]

@sandersaares This PR is for 2.1, but you may want to try it out.

[dsparacio]

Fixed with merged PR #1109