icbaker
commented
5 months ago Thanks for the content link, I was able to repro the same issue with similar stack traces to the two root causes shown in your log snippet:
Caused by: org.xmlpull.v1.XmlPullParserException: Dangling < (position:TEXT (whitespace)@22:1 in >java.io.InputStreamReader@411f85f) at com.android.org.kxml2.io.KXmlParser.peekType(KXmlParser.java:1002) at com.android.org.kxml2.io.KXmlParser.next(KXmlParser.java:448) at com.android.org.kxml2.io.KXmlParser.next(KXmlParser.java:313) at androidx.media3.extractor.text.ttml.TtmlParser.parseToLegacySubtitle(TtmlParser.java:208)Caused by: java.lang.NullPointerException at androidx.media3.common.util.Assertions.checkNotNull(Assertions.java:155) at androidx.media3.extractor.text.ttml.TtmlParser.parseToLegacySubtitle(TtmlParser.java:211)
I added logging to see what the sample data looked like when these exceptions were being thrown:
if (ttmlSubtitle == null) {
throw new NullPointerException("data=" + new String(data, Charsets.UTF_8));
}
return checkNotNull(ttmlSubtitle);
} catch (XmlPullParserException xppe) {
throw new IllegalStateException(
"Unable to decode source\ndata=" + new String(data, Charsets.UTF_8), xppe);
}I saw that the data does indeed look incomplete (I've removed the subtitle text manually, just leaving the XML structure) - see the missing tt> at the end:
<?xml version="1.0" encoding="utf-8"?>
<tt xml:lang="en" xmlns:tt="http://www.w3.org/ns/ttml"
xmlns:ttp="http://www.w3.org/ns/ttml#parameter"
xmlns="http://www.w3.org/ns/ttml"
xmlns:tts="http://www.w3.org/ns/ttml#styling"
ttp:contentProfiles="http://www.w3.org/ns/ttml/profile/imsc1.1/text">
<head>
<styling>
<style xml:id="s0" tts:fontStyle="normal" tts:fontFamily="sansSerif" tts:fontSize="70%" tts:lineHeight="normal"
tts:color="#FFFFFF" tts:wrapOption="noWrap" tts:textAlign="center"/>
</styling>
<layout>
<region xml:id="r0" tts:origin="15% 80%" tts:extent="70% 20%" tts:overflow="visible" tts:displayAlign="center"/>
</layout>
</head>
<body style="s0">
<div region="r0">
<p begin="5544:22:54.620" end="5544:22:57.119"><span tts:textAlign="center">foo<br /></span></p>
<p begin="5544:22:57.260" end="5544:22:59.759"><span tts:textAlign="center">bar<br /></span></p>
</div>
</body>
</??????????I also curl'd the .m4s file that provides this sample so I could inspect it directly. If you want to look it up, it's the one that ends 1663315.m4s.
I analyzed this file using a hex viewer and an MP4 analyzer. I found that:
- The TTML data is complete in the file, it ends with
3C 2F 74 74 3E 0Awhich is</tt>\n - The
mdatbox starts at offset0xB0 = 176and declares a size of0x0407 = 1031- This reaches all the way to the end of the file (just after the final
\nchar)
- This reaches all the way to the end of the file (just after the final
- However the
trunbox also declares the sample position and length in the file. It says:sample_count = 1data_offset = 108sample_size = 1019
Consulting ISO 14496-12:2012 section 8.8.8.3 which defines the semantics of the trun box, it says:
data_offsetis added to the implicit or explicitdata_offsetestablished in the track fragment header.
The 'track fragment header' is the tfhd which does not set an explicit data_offset, since flags = 131072 and the base-data-offset-present flag is 0x000001 - this means an implicit data_offset is used as described in section 8.8.7.1:
If not provided, the base-data-offset for the first track in the movie fragment is the position of the first byte of the enclosing Movie Fragment Box,
The Movie Fragment Box is the moof box, which is at position 0x4C = 76 in this file.
Therefore the sample data offset from the start of the file is 108 + 76 = 184 = 0xB8. This matches up with looking at the file in a hex editor, because at that address is the first < of <?xml version="1.0".
However, the sample_size from the trun box indicates the sample ends at 184 + 1019 = 1203 = 0x4B3. Looking back to the hex editor, this truncates the XML to: [...]</body>\n</. This matches up with the truncated data I showed above and also explains why the XML parser is complaining about a Dangling <.
Conclusion: This media file is malformed, because the trun box truncates the sample data before the end of the sample - which results in ExoPlayer passing an incomplete TTML snippet to its TTML parsing machinery, which (rightly) fails to parse it.
Version
Media3 1.3.1
More version details
Current release branch from the current Media3 project was used, which should be 1.3.1 at present time. Although the issue can be reproduced in older versions of ExoPlayer demo and ExoPlayer.
Devices that reproduce the issue
Android TV Emulator API 29
Devices that do not reproduce the issue
No response
Reproducible in the demo app?
Yes
Reproduction steps
A detailed account of events:
Hello ExoPlayer team,
This is a bug report regarding the DASH (MPD) subtitles in ExoPlayer.
We tested the MPD subtitles in ExoPlayer (current version, although it can be reproduced in older versions as well), and found an inconsistency with XML either not parsing, bailing on the tt tag, or just unable to decode the subtitles completely. Other times there is no visible error in the logs, however subtitles are still not shown (when selected through the subtitle control in the player (the stream URLs where loaded into the media.exolist.json (in ExoPlayer Demo - androidx.media3).
The subtitle specification is as follows: mimeType="application/mp4" codecs="stpp.ttml.im1t"
This is a subtitle portion excerpt from the stream file as well. We can ofcourse provide you with a private URL stream link on email for testing this issue, or anything that you might need.
After the subtitle is selected (using the ExoPlayer controls in Demo), they are indeed selected as seen in the EvenLogger log:
The errors which can be seen through the logs are as follows:
The error message "Unexpected decode error" is from SimpleSubtitleDecoder.java (in ExoPlayer - androidx.media3) after XML parsing fails:
Which points to the code in function parseToLegacySubtitle(byte[] data, int offset, int length) in TtmlParser.java (in ExoPlayer - androidx.media3):
This is an excerpt from this java file for the setting of ttmlSubtitle object and the error messages we get above, which get triggered after a parsing exception.
The object ttmlSubtitle cannot be set (because the TtmlNode.TAG_TT, cannot be found apparently for some reason, and thus the object remains null) and therefore everything else fails, or the "Unable to decode source" gets called after, as we see above in the error messages.
The second error message as parsing continues:
Caused by: java.lang.IllegalStateException: Unable to decode sourceAnd also, below that:Caused by: org.xmlpull.v1.XmlPullParserException: Dangling < (position:TEXT (whitespace)@22:1 in java.io.InputStreamReader@c716f03)This caused by parsing exception is not always the same it varies by parsed element of course, so it can sometimes be something different, like for example:Caused by: org.xmlpull.v1.XmlPullParserException: expected: '>' actual: '' (position:END_TAG </body>@23:8 in java.io.InputStreamReader@34e833f)And then it just keeps returning the same error messages (and caused by cases) as parsing continues or the subtitle remains selected and the stream continues running.
There is also a second case that we get, where there is no error at all (in the logs or noticed otherwise) but the subtitles are still not showing, when the subtitle is selected.
Can you kindly take a look at this issue, if there is something we are missing here, please enlighten us. As I said previously, a URL stream link will be provided for you on mentioned email address for testing and debugging this issue, as we cannot give out the links publicly.
Thank you!
Expected result
The subtitles are showing during the stream play.
Actual result
First case: No subtitles are showing with error messages in logcat. Second case: No subtitles are showing with no error messages in logcat.
Media
We will provide you with a private URL stream link on mentioned email address for testing and debugging this issue.
Bug Report
adb bugreportto android-media-github@google.com after filing this issue.