Support HLS w/ MP4 subtitles without MIME type info

[Romain-Bernier]

Have you read the FAQ and checked for duplicate open issues? Yes

What version of Shaka Player are you using? 2.5.1-master

Can you reproduce the issue with our latest release version? yes

Can you reproduce the issue with the latest code from master? Yes

Are you using the demo app or your own custom app? Both

If custom app, can you reproduce the issue using our demo app? Yes

What browser and OS are you using? Windows 10 & Chrome Version 74.0.3729.169

For embedded devices (smart TVs, etc.), what model and firmware version are you using? N/A

What are the manifest and license server URIs? https://irtdashreference-i.akamaihd.net/dash/live/901161/keepixo1/playlistBR2.m3u8 No DRM

What did you do? I want to read the content with the subtitles, they are not working, I don't understand why.

What did you expect to happen? The subtitles working properly

What actually happened? When the subtitles are activated, I can't see them on the video.

[TheModMaker]

We need to detect the MIME type and codec of the streams when we parse the manifest. We usually use the info in the manifest to determine this, but HLS doesn't always include this info. In this case, you are using either TTML or VTT embedded in MP4 and we can't detect which based on the manifest. Since we don't know the codec (and which parser to use), we ignore that stream assuming it is unsupported.

[Romain-Bernier]

Ok, thanks for the reply

[ismena]

@Romain-Bernier While we're working on fixing it, one possible work-around: If you only have one type of subtitles in your content (either ttml in mp4 or vtt in mp4) and you know which, you can run shaka.text.TextEngine.registerParser( 'application/mp4', shaka.text.Mp4TtmlParser); or shaka.text.TextEngine.registerParser( 'application/mp4', shaka.text.Mp4VttParser);

before loading your content and it should display the subtites ;)

[kqyang]

There is a related bug on Shaka Packager: https://github.com/google/shaka-packager/issues/645.

HLS Authoring Specification for Apple Devices specifies that

5.10. The kind of subtitles SHOULD be specified in the CODECS attribute of the associated EXT-X-STREAM-INF tags. You SHOULD use “stpp.ttml.im1t” to identify IMSC1 subtitles. You MAY use “wvtt" to identify WebVTT subtitles.

So the codec information for WebVTT is optional, but recommended for TTML.

Would it work in the player to properly support TTML codec (stpp.ttml.im1t) and WebVTT codec (wvtt) if they are present and assume webvtt subtitles if the codec information is missing (since codec information for TTML should be present)?

[joeyparrish]

That seems reasonable to me. We would be happy to review a PR if anyone has time to work on this. Thanks!

[johndeu]

Following this issue also, as I can't get any of the live streams using HLS and MP4/IMSC1 (ISO 14496 part 30) fragments to render in the Shaka player at all.

Having the same issue with this HLS manifest. We can share some live stream samples as well. I'm asking my team if they can possibly take this PR @joeyparrish

EXTM3U

EXT-X-VERSION:7

EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="audio",DEFAULT=YES,CHANNELS="2",URI="QualityLevels(128000)/Manifest(audio,format=m3u8-cmaf)"

EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Auto-generated",LANGUAGE="en-us",DEFAULT=YES,AUTOSELECT=YES,CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog",URI="QualityLevels(4800)/Manifest(Auto-generated,format=m3u8-cmaf)"

EXT-X-STREAM-INF:BANDWIDTH=2702136,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.5,stpp.ttml.im1t",AUDIO="audio",SUBTITLES="subs"

QualityLevels(2500000)/Manifest(video,format=m3u8-cmaf)

EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=2702136,RESOLUTION=1280x720,CODECS="avc1.64001f",URI="QualityLevels(2500000)/Manifest(video,format=m3u8-cmaf,type=keyframes)"

EXT-X-STREAM-INF:BANDWIDTH=138976,CODECS="mp4a.40.5",AUDIO="audio",SUBTITLES="subs"

QualityLevels(128000)/Manifest(audio,format=m3u8-cmaf)