search

androidx / media

Jetpack Media3 support libraries for media use cases, including ExoPlayer, an extensible media player for Android
https://developer.android.com/media/media3
Apache License 2.0
1.68k stars 399 forks source link

Unable to get CEA-608 text tracks to appear in HLS live stream. #1446

Closed hudsonensemble closed 4 months ago

hudsonensemble commented 4 months ago

I am currently trying to get my HLS live stream to display captions in CEA-608 format but when viewing the EventLogger I do not see them appear. I have tested the livestream on https://www.theoplayer.com/test-your-stream-hls-dash-hesp and was able to enable the captions "CC1".

When testing on the demo app I do not see any captions available. Here are the logs from the EventLogger on the demo app.

2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D tracks [eventTime=0.21, mediaPos=2.05, window=0, period=0 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D group [ 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:0, id=0, mimeType=video/avc, container=application/x-mpegURL, bitrate=3211421, codecs=avc1.4d001f, res=960x540, fps=30.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:1, id=1, mimeType=video/avc, container=application/x-mpegURL, bitrate=5173671, codecs=avc1.64001f, res=1280x720, fps=30.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:2, id=2, mimeType=video/avc, container=application/x-mpegURL, bitrate=8017171, codecs=avc1.640028, res=1920x1080, fps=30.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:3, id=3, mimeType=video/avc, container=application/x-mpegURL, bitrate=378203, codecs=avc1.4d000d, res=256x144, fps=15.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:4, id=4, mimeType=video/avc, container=application/x-mpegURL, bitrate=638171, codecs=avc1.4d000d, res=416x234, fps=30.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:5, id=5, mimeType=video/avc, container=application/x-mpegURL, bitrate=1354921, codecs=avc1.4d001e, res=640x360, fps=30.0, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D ] 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D group [ 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:0, id=0, mimeType=audio/mp4a-latm, container=application/x-mpegURL, codecs=mp4a.40.5, language=en, labels=[androidx.media3.common.Label@be3f2307], selectionFlags=[auto,default], supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D ] 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D group [ 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D [X] Track:0, id=ID3, mimeType=application/id3, supported=YES 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D ] 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D Metadata [ 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D HlsTrackMetadataEntry 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D ] 2024-06-11 17:18:55.146 3473-3473 EventLogger androidx.media3.demo.main D ]

The HLS manifest has no mention of the closed_caption attribute. Is this required for exoplayer to understand what text tracks are available?

Here is what the manifest looks like:

EXTM3U

EXT-X-VERSION:5

EXT-UPLYNK-LIVE

EXT-X-START:TIME-OFFSET=0.00

EXT-X-INDEPENDENT-SEGMENTS

EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="Primary",LANGUAGE="en",AUTOSELECT=YES,DEFAULT=YES

UPLYNK-MEDIA0:960x540x30,main-31,2x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=960x540,BANDWIDTH=3480203,CODECS="mp4a.40.5,avc1.4d001f",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=1878794

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/d.m3u8

UPLYNK-MEDIA0:1280x720x30,high-31,2x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=1280x720,BANDWIDTH=5595203,CODECS="mp4a.40.5,avc1.64001f",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=3141667

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/e.m3u8

UPLYNK-MEDIA0:1920x1080x30,high-40,2x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=1920x1080,BANDWIDTH=8426953,CODECS="mp4a.40.5,avc1.640028",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=4563634

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/f.m3u8

UPLYNK-MEDIA0:256x144x15,main-13,1x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=256x144,BANDWIDTH=389953,CODECS="mp4a.40.5,avc1.4d000d",FRAME-RATE=15.000,AUDIO="aac",AVERAGE-BANDWIDTH=223596

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/a.m3u8

UPLYNK-MEDIA0:416x234x30,main-13,2x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=416x234,BANDWIDTH=648453,CODECS="mp4a.40.5,avc1.4d000d",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=388543

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/b.m3u8

UPLYNK-MEDIA0:640x360x30,main-30,2x48000

EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=640x360,BANDWIDTH=1412203,CODECS="mp4a.40.5,avc1.4d001e",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=809330

https://XXXXXXXXXXXXXX.com/channel/XXXXXXXXXXXXXX/c.m3u8

icbaker commented 4 months ago

If your stream doesn't contain TYPE=CLOSED-CAPTIONS as described in https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis#section-4.4.6.1 ExoPlayer by default assumes that CEA-608 captions are not present.

If you can't fix the playlist, you can change ExoPlayer's behaviour by either providing a customized TsExtractor (to control the metadata of the exposed subtitle track) or set exposeCea608WhenMissingDeclarations = true (to expose a track called 'unknown') when instantiating a DefaultHlsExtractorFactory. Either way you will need to plumb this custom HlsExtractorFactory into an HlsMediaSource.Factory and customize your player with this. More info: https://developer.android.com/guide/topics/media/exoplayer/media-sources#customizing-media-source-creation

hudsonensemble commented 4 months ago

@icbaker Thank you for your reply. One other question, If I don't change the playlist to include TYPE=CLOSED-CAPTIONS can I still enable chunckless preparation?

icbaker commented 4 months ago

If you make either the custom TsExtractor or exposeCea608WhenMissingDeclarations change I suggested above, it should still work with chunkless preparation enabled.

icbaker commented 4 months ago

Apologies, I was wrong above, chunkless preparation is incompatible with undeclared captions - so you will need to disable it in order to access subtitles in streams like this.