TsExtractor: select PesReader based on HDMV format identifier

Avetri commented 3 years ago

Clear reproduction steps including observed and expected behavior

Download M2TS_DTSHD_AC3_MP2_MP3_DTS.ts and put it on your HTTP server. Add URI for M2TS_DTSHD_AC3_MP2_MP3_DTS.ts file to the media.exolist.json. Run ExoPlayer demo app and try to play the file.

URI to test content for reproduction

Sample file: M2TS_DTSHD_AC3_MP2_MP3_DTS.ts

ExoPlayer version number

2.13.2 (Any I suspect.) SHA1 ID 4364b9150f236813a2070fc79d3bcb76e11969ac

Android version

7

Android device

Any.

Tracks in logcat:
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger: tracks [eventTime=0.77, mediaPos=0.00, window=0, period=0
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   MediaCodecVideoRenderer [
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     Group:0, adaptive_supported=N/A [
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=1/4113, mimeType=video/avc, codecs=avc1.640029, res=1920x1080, supported=YES
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   ]
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   LibvpxVideoRenderer []
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   Libgav1VideoRenderer []
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   MediaCodecAudioRenderer [
2021-05-07 14:25:20.026 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     Group:0, adaptive_supported=N/A [
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=1/4353, mimeType=audio/ac3, channels=6, sample_rate=48000, language=en, supported=YES
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     Group:1, adaptive_supported=N/A [
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:0, id=1/4354, mimeType=audio/mpeg, channels=2, sample_rate=48000, language=en, supported=YES
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     Group:2, adaptive_supported=N/A [
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:0, id=1/4355, mimeType=audio/mpeg-L2, channels=2, sample_rate=48000, language=en, supported=YES
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   ]
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   LibopusAudioRenderer []
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   LibflacAudioRenderer []
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   FfmpegAudioRenderer []
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   TextRenderer []
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   MetadataRenderer [
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     Group:0, adaptive_supported=N/A [
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=null, mimeType=application/x-scte35, supported=YES
2021-05-07 14:25:20.027 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 14:25:20.028 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   ]
2021-05-07 14:25:20.028 11471-11471/com.google.android.exoplayer2.demo D/EventLogger:   CameraMotionRenderer []
2021-05-07 14:25:20.028 11471-11471/com.google.android.exoplayer2.demo D/EventLogger: ]

Exception in SCTE35 parser:
2021-05-07 14:25:22.500 11471-11834/com.google.android.exoplayer2.demo E/ExoPlayerImplInternal: Playback error
      com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:586)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:154)
        at android.os.HandlerThread.run(HandlerThread.java:61)
     Caused by: java.lang.IllegalStateException
        at com.google.android.exoplayer2.util.Assertions.checkState(Assertions.java:86)
        at com.google.android.exoplayer2.util.ParsableBitArray.assertValidOffset(ParsableBitArray.java:349)
        at com.google.android.exoplayer2.util.ParsableBitArray.readBits(ParsableBitArray.java:189)
        at com.google.android.exoplayer2.metadata.scte35.SpliceInfoDecoder.decode(SpliceInfoDecoder.java:65)
        at com.google.android.exoplayer2.metadata.SimpleMetadataDecoder.decode(SimpleMetadataDecoder.java:34)
        at com.google.android.exoplayer2.metadata.MetadataRenderer.render(MetadataRenderer.java:135)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:947)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:477)
        at android.os.Handler.dispatchMessage(Handler.java:98) 
        at android.os.Looper.loop(Looper.java:154) 
        at android.os.HandlerThread.run(HandlerThread.java:61) `

Avetri commented 3 years ago

Real tracks:

Audio 1 ID : 4352 (0x1100) Format : DTS XLL Format/Info : Digital Theater Systems Commercial name : DTS-HD Master Audio

Audio 2 ID : 4353 (0x1101) Format : AC-3 Format/Info : Audio Coding 3 Commercial name : Dolby Digital

Audio 3 ID : 4354 (0x1102) Format : MPEG Audio Format version : Version 1 Format profile : Layer 3 Format settings : Joint stereo / MS Stereo

Audio 4 ID : 4355 (0x1103) Format : MPEG Audio Format version : Version 1 Format profile : Layer 2

Audio 5 ID : 4356 (0x1104) Format : DTS Format/Info : Digital Theater Systems

Avetri commented 3 years ago

PMT Registration Tag (0x05) Format Identifier is HDMV.

christosts commented 3 years ago

@kim-vde can you take a look?

andrewlewis commented 3 years ago

Please see #5330 where we changed the default behavior for this stream type to be handling it as SCTE. You can probably 'fix' this by setting FLAG_ENABLE_HDMV_DTS_AUDIO_STREAMS.

Avetri commented 3 years ago

Please see #5330 where we changed the default behavior for this stream type to be handling it as SCTE. You can probably 'fix' this by setting FLAG_ENABLE_HDMV_DTS_AUDIO_STREAMS.

Thanks.

It detects (in ExoPlayer demo at least) a DTS audio track now:

Group:0, adaptive_supported=N/A [
2021-05-07 20:46:25.059 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=1/4356, mimeType=audio/vnd.dts, bitrate=1536000, channels=6, sample_rate=48000, language=en, supported=YES
2021-05-07 20:46:25.059 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:     ]

But there are two DTS tracks. And it still craches on SCTE:

2021-05-07 20:46:25.060 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:   MetadataRenderer [
2021-05-07 20:46:25.060 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:     Group:0, adaptive_supported=N/A [
2021-05-07 20:46:25.060 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=null, mimeType=application/x-scte35, supported=YES
2021-05-07 20:46:25.060 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:     ]
2021-05-07 20:46:25.060 27654-27654/com.google.android.exoplayer2.demo D/EventLogger:   ]
2021-05-07 20:46:27.424 27654-28014/com.google.android.exoplayer2.demo E/ExoPlayerImplInternal: Playback error
      com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:586)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:154)
        at android.os.HandlerThread.run(HandlerThread.java:61)
     Caused by: java.lang.IllegalStateException
        at com.google.android.exoplayer2.util.Assertions.checkState(Assertions.java:86)
        at com.google.android.exoplayer2.util.ParsableBitArray.assertValidOffset(ParsableBitArray.java:349)
        at com.google.android.exoplayer2.util.ParsableBitArray.readBits(ParsableBitArray.java:189)
        at com.google.android.exoplayer2.metadata.scte35.SpliceInfoDecoder.decode(SpliceInfoDecoder.java:65)
        at com.google.android.exoplayer2.metadata.SimpleMetadataDecoder.decode(SimpleMetadataDecoder.java:34)
        at com.google.android.exoplayer2.metadata.MetadataRenderer.render(MetadataRenderer.java:135)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:947)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:477)
        at android.os.Handler.dispatchMessage(Handler.java:98) 
        at android.os.Looper.loop(Looper.java:154) 
        at android.os.HandlerThread.run(HandlerThread.java:61)

kim-vde commented 3 years ago

The stream you provided has:

A track with stream type 0x82 (ID 4356), which can be SCTE subtitle or DTS 6 channel audio (see https://en.wikipedia.org/wiki/Program-specific_information). Setting flag FLAG_ENABLE_HDMV_DTS_AUDIO_STREAMS tells ExoPlayer that the track is DTS (and not SCTE).
A track with stream type 0x86 (ID 4352), which can be SCTE-35 splice info or DTS 8 channel lossless audio (see https://en.wikipedia.org/wiki/Program-specific_information). ExoPlayer currently assumes that the track is of type SCTE-35. The track can be ignored though by setting flag FLAG_IGNORE_SPLICE_INFO_STREAM.

Would it work for you to set flag FLAG_IGNORE_SPLICE_INFO_STREAM as well?

Avetri commented 3 years ago

@ojw28 , thanks for a cleaning up the mess.

@kim-vde , thank You. Flags (as a solution) don't seem right. Why does ExoPlaye use external flags instead of descriptors from PMT?

kim-vde commented 3 years ago

In the stream you provided, the PMT contains a program descriptor which is a registration descriptor (descriptor_tag is 5). This registration descriptor has format_identifier HDMV, which means that the stream conforms to the spec of "System Description Blu-ray Disc Read-Only Format part 3 Audio Visual Basic Specifications".

Do you mean that we should deduct from there that the elementary streams of type 0x82 and 0x86 are DTS streams? Could you elaborate why this is always true?

Avetri commented 3 years ago

PMT Registration Tag (0x05) with "HDMV" format identifier with stream types from 0x81 to 0x87, PMT Registration Tag (0x05) with "AC-3" or "EAC3" format identifiers, DTS PMT descriptor (0x7B), AC3 PMT descriptor (0x6A) and E-AC3 PMT descriptor (0x7A) should lead to audio streams according to standards because they are standards. But ofcourse nodody can garant from a wrong stream.

I don't insist that somebody has to implement the player in this way. I just only asked a question:

Flags (as a solution) don't seem right. Why does ExoPlaye use external flags instead of descriptors from PMT?

Issue #8911 isn't actual [not completely but ...] for me now after your help. So I closed it.

kim-vde commented 3 years ago

I marked this issue as an enhancement.

Here is a summary of the current situation:

PMT Registration Tag (0x05) with "AC-3" or "EAC3" format identifiers --> Stream type already deduced from this here.
DTS PMT descriptor (0x7B), AC3 PMT descriptor (0x6A) and E-AC3 PMT descriptor (0x7A) --> Stream type already deduced from this here.
PMT Registration Tag (0x05) with "HDMV" format identifier with stream types from 0x81 to 0x87 --> Stream types 0x82 and 0x86 are ambiguous. We currently select the correct one based on external flags. We could probably set the PesReader to a DtsReader by default in case the format identifier is "HDMV".

google / ExoPlayer

TsExtractor: select PesReader based on HDMV format identifier #8911