Possible Bug: Load WebVTT (.vtt) subtiles in Dynamic Live MPD via SegmentTemplate

[MBS9]

Hello all!

I have an Live Dynamic MPD which includes the following tags:

<AdaptationSet codecs="wvtt" mimeType="text/vtt" contentType="text" lang="en">
    <Representation id="caption_zxx" bandwidth="256">
        <SegmentTemplate timescale="1000000" duration="10000000" media="{linkToWebVTTFile}"/>
    </Representation>
</AdaptationSet>

When this tag is removed everything works well (apart of course that the subtitles are not displayed), but when this tag is added dash.js says Error creating source buffer of type : text.

I traced this bug and found that if I change this if statement from if (codec.match(/application\/mp4;\s*codecs="(stpp|wvtt).*"/i)) to if (codec.match(/application\/mp4;\s*codecs="(stpp|wvtt).*"/i) || codec.match(/text/))) the issue resolves and the subtitles are displayed correctly.

It seems like WebVTT (.vtt) files require a text buffer, but when this buffer is created, dash.js doesn't create a text buffer but instead uses the same buffer which would be used for audio/video (which does not support text). This seems to cause an error. That above linked statement seems to filter out the codecs which require this text buffer (which are subtitles), but it doesn't catch the vtt since its MIME Type is text/vtt not application/mp4. By adding the check for the codec to match the regex "text", the vtt subtitles (since our file has MIME Type text/vtt) are now caught and the correct buffer is created for it.

If the dash.js team agrees that this is a bug, I would love to open a PR to fix this.

[dsilhavy]

Can you share the stream? Does this link to an external WebVTT file or is WebVTT included in ISOBMFF segments?

[MBS9]

Hello @dsilhavy!

It links to external WebVTT file

[MBS9]

Thanks for your help.

[dsilhavy]

Our reference content for external WebVTT files does not define a codecs attribute: https://dash.akamaized.net/akamai/test/caption_test/ElephantsDream/elephants_dream_480p_heaac5_1_https.mpd

I took a brief look at the IOP guidelines, and they only mention the codecs attribute for embedded texttracks: If @mimeType is set to "application/mp4", then this element shall be present

I need to check this in more detail, but probably your content would work as exptected if you remove codecs="wvtt"

[MBS9]

Hello @dsilhavy!

Thanks for your quick response and help with fixing this.

I updated the MPD to not include the codecs="wvtt" attribute. Unfortunately, it still produces the same error.

[dsilhavy]

@MBS9 I looked at your MPD. I think the problem is that you are using <SegmentTemplate> in which case dash.js assumes that you are providing a subtitle which is embedded in ISOBMFF segments. Creating the corresponding SourceBuffer then fails.

As you are linking to an external subtitle file in your MPD this format should work:

<AdaptationSet mimeType="text/vtt" lang="en">
<Representation id="caption_zxx" bandwidth="256">
        <BaseURL>https://us-central1-envelop-stream-dev.cloudfunctions.net/getDynamicVtt?url=https%3A%2F%2Fcompositions.s3.us-west-2.amazonaws.com%2Fenvelop%2Fsmaller-Sexoid%2520E4L%2Foutput.vtt&startTimeSecs=1295160</BaseURL> 
</Representation>
</AdaptationSet>

Can you modify the MPD accordingly? In the catch block of SourceBufferSink we will handle this then as the textrack is no longer embedded.

[MBS9]

Hello @dsilhavy

Thanks for your help! I updated the MPD as requested. Unfortunately, it still doesn't quite do what we want it to do.

While the error is no longer present, as new periods are added to the Dynamic MPD the subtitles don't work on the newly added periods. They only work when we are playing a period which was present when the MPD was first requested.

So, for example, if the MPD has periods 1 and 2 when first being requested, then periods 1 and 2 will play without issues. When more periods are added after the stream has started (say period 3 is also added), then for period 3, the audio will play but the subtitles won't. Same happens for periods 4,5,6..... as they are added.

(In case this affects the debugging: We remove the old periods once they finished playing to keep the size of the MPD smaller)

Thanks, a lot, in advance!

[MBS9]

Hello @dsilhavy !

I am now going to revert the MPD back to using the SegmentTemplate tag instead of BaseURL.

Below are 3 pastebins of the MPD before the revert (so with BaseURL tags). They show how the MPD looked like in different points in time (as Periods were being added/removed):

1. https://pastebin.com/p1svq8SY
2. https://pastebin.com/MvjaxY2m
3. https://pastebin.com/Ku3nyFrA

Thanks!

[dsilhavy]

@MBS9 Can you check if your SegmentTemplate based format now works in https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html. I did small change in #4265

[MBS9]

Hello @dsilhavy !

The SegmentTemplate based format does work now. Thanks for your help!

Let me know if with this change in the reference client we should consider this issue resolved and close it - on my side it is ready to be closed as my problem is now resolved.

Thanks for your help again!

[dsilhavy]

Perfect, thanks for testing. I close this issue then.