Are TTML subtitles supported in HLS?

[RicFlinn]

I have a stream that consists of fragmented MPEG-4 segments (separate audio and video segments), and also includes TTML/IMSC1 subtitle segments, also fMP4. I have a multivariant playlist and media playlists for each stream. Video and audio play as expected, and it appears the subtitle segments are being consumed by ExoPlayer, but I don't see anything on the screen. I do see subtitles with other input formats (including DASH with the same input streams), just not with HLS+TTML.

I see this message in Logcat: Text Track 0 Lang=en codec=stpp.ttml.im1t Label=eng which seems to indicate the track is properly selected.

I'm using ExoPlayer 2.18.2.

[tonihei]

Based on your LogCat output, I assume ExoPlayer correctly detects and parses the subtitle track. (Is this generated by ExoPlayer's EventLogger or some other class?)

If you can't see the subtitles, it might be that they are not selected (this should be visible in the EventLogger output by the [x] or [ ] in front of the tracks) or there is an issue with subtitle parsing or their timestamps. For the latter kind of issue, you'd need to make the test stream available to us for further investigation. You can post a link here or send an email to dev.exoplayer@gmail.com with "Issue #10981" in the title and then comment here that you've done that.

And to answer the question in the issue title: Yes, TTML subtitles are support by HLS.

[RicFlinn]

Thanks, at least I know it should (in theory) work. It could certainly be something I'm missing or configuring wrong.

I enabled EventLogger and it does appear the subtitle track is selected. When enabling subtitles I see this: group [ [X] Track:0, id=sub:eng, mimeType=application/ttml+xml, codecs=stpp.ttml.im1t, ] along with the video and audio tracks.

I added an onCues listener, but it remains silent.

The app I'm working on also supports DASH, and I've been able to serve identical streams via both DASH and HLS, and the DASH implementation shows the TTML subtitles, so I suspect the fragments are formatted correctly.

One other interesting thing, on some test streams I do see some warnings from TtmlDecoder that say Ignoring region without an origin. But, I see these same warnings with DASH as well, and in that case the subtitles display on the screen (an in onCues).

I put together a sample that hopefully you can use to test this. I bundled up some audio/video/subtitle segments and playlists. I've just been using the ExoPlayer demo app to play it, streaming from a local web server. I do see the requests for the subtitle segments, but nothing on screen. I also included a static DASH manifest that I can also play via the demo app, and it does show subtitles. Here's a link: https://1drv.ms/u/s!Ar-5IRkcwwk9gdBJbyhdkVZOqNCJCw?e=2NiD2s

[RicFlinn]

I had a chance to look into this a bit more, and I think I know what's going on.

The streams I'm looking at are HLS streams that were converted from live DASH streams. DASH, of course, is very dependent on clock time for playback of live streams to work, including subtitles, and these subtitles have 'begin' and 'end' times relative to the time the streams were delivered. So the TtmlDecoder is likely correctly looking at subtitles that should have been displayed last week or last month, relative to current clock time.

So now the question is, can I do anything about this. With DASH it wasn't terribly difficult; I just added a UTCTiming element to the manifest with a time that matched the stream recording, and convinced ExoPlayer to use that time as its reference. Is there any such ability in ExoPlayer for HLS?

[tonihei]

Is there any such ability in ExoPlayer for HLS?

HLS doesn't have a similar concept. You can add #EXT-X-PROGRAM-DATE-TIME tags to indicate the real time at a specific point in the playlist, but I'm not sure if it had any practical influence on the subtitle timestamp handling.