search

MediaArea / MediaInfoLib

Convenient unified display of the most relevant technical and tag data for video and audio files.
https://mediaarea.net/MediaInfo
BSD 2-Clause "Simplified" License
636 stars 176 forks source link

TTML Files Missing TimeCode_FirstFrame and TimeCode_LastFrame #2112

Closed joe-sciame-wm closed 2 months ago

joe-sciame-wm commented 3 months ago

Hi Jerome,

Noticed a timed text file is missing the TimeCode_FirstFrame & TimeCode_LastFrame from the XML output. Is there a way to unify the First and Last Frame across formats? We are using code to generate the the timecode policy to determine if the text is within the in/out of the composition.

Other text files i.e. SCC files have:

<TimeCode_FirstFrame>01:00:04:21</TimeCode_FirstFrame>
<TimeCode_LastFrame>01:54:57:29</TimeCode_LastFrame>

Instead I'm seeing the outputs for TTML have

<Duration_Start>3.378</Duration_Start>
<Duration_Start_String>3 s 378 ms</Duration_Start_String>
<Duration_Start_String1>3 s 378 ms</Duration_Start_String1>
<Duration_Start_String2>3 s 378 ms</Duration_Start_String2>
<Duration_Start_String3>00:00:03.378</Duration_Start_String3>
<Duration_Start_String4>00:00:03:09</Duration_Start_String4>
<Duration_Start_String5>00:00:03.378 (00:00:03:09)</Duration_Start_String5>
<Duration_End>126.584</Duration_End>
<Duration_End_String>2 min 6 s</Duration_End_String>
<Duration_End_String1>2 min 6 s 584 ms</Duration_End_String1>
<Duration_End_String2>2 min 6 s</Duration_End_String2>
<Duration_End_String3>00:02:06.584</Duration_End_String3>
<Duration_End_String4>00:02:06:11</Duration_End_String4>
<Duration_End_String5>00:02:06.584 (00:02:06:11)</Duration_End_String5>

Generated policy from code:

<policy name="TimeCode Checks" type="and" level="error">
    <rule name="Starting Timecode" occurrence="*" operator="&gt;=" tracktype="Text" level="error" value="TimeCode_FirstFrame">00:59:56:12</rule>
    <rule name="End Timecode" occurrence="*" operator="&lt;=" tracktype="Text" level="error" value="TimeCode_LastFrame">01:01:56:07</rule>
</policy>

Full MediaInfo

<?xml version="1.0" encoding="UTF-8"?>
<MediaInfo
    xmlns="https://mediaarea.net/mediainfo"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="https://mediaarea.net/mediainfo https://mediaarea.net/mediainfo/mediainfo_2_0.xsd"
    version="2.0">
<creatingLibrary version="24.06.20240806" url="https://mediaarea.net/MediaInfo">MediaInfoLib</creatingLibrary>
<media ref="/Users/jsciame/Downloads/DeepwaterHorizon_ES419_Subtitles.xml">
<track type="General">
<Count>350</Count>
<StreamCount>1</StreamCount>
<StreamKind>General</StreamKind>
<StreamKind_String>General</StreamKind_String>
<StreamKindID>0</StreamKindID>
<TextCount>1</TextCount>
<Text_Format_List>TTML</Text_Format_List>
<Text_Format_WithHint_List>TTML</Text_Format_WithHint_List>
<Text_Codec_List>TTML</Text_Codec_List>
<Text_Language_List>Spanish (Latin America)</Text_Language_List>
<CompleteName>/Users/jsciame/Downloads/DeepwaterHorizon_ES419_Subtitles.xml</CompleteName>
<FolderName>/Users/jsciame/Downloads</FolderName>
<FileNameExtension>DeepwaterHorizon_ES419_Subtitles.xml</FileNameExtension>
<FileName>DeepwaterHorizon_ES419_Subtitles</FileName>
<FileExtension>xml</FileExtension>
<Format>TTML</Format>
<Format_String>TTML</Format_String>
<Format_Info>Timed Text Markup Language</Format_Info>
<Format_Url>https://en.wikipedia.org/wiki/Timed_Text_Markup_Language</Format_Url>
<Format_Extensions>dfxp ttml</Format_Extensions>
<Format_Commercial>TTML</Format_Commercial>
<Format_Profile>SMPTE-TT</Format_Profile>
<FileSize>194113</FileSize>
<FileSize_String>190 KiB</FileSize_String>
<FileSize_String1>190 KiB</FileSize_String1>
<FileSize_String2>190 KiB</FileSize_String2>
<FileSize_String3>190 KiB</FileSize_String3>
<FileSize_String4>189.6 KiB</FileSize_String4>
<Duration>6424.383</Duration>
<Duration_String>1 h 47 min</Duration_String>
<Duration_String1>1 h 47 min 4 s 383 ms</Duration_String1>
<Duration_String2>1 h 47 min</Duration_String2>
<Duration_String3>01:47:04.383</Duration_String3>
<Duration_String4>01:46:57:23</Duration_String4>
<Duration_String5>01:47:04.383 (01:46:57:23)</Duration_String5>
<OverallBitRate>242</OverallBitRate>
<OverallBitRate_String>242 b/s</OverallBitRate_String>
<FrameRate>23.976</FrameRate>
<FrameRate_String>23.976 (24000/1001) FPS</FrameRate_String>
<FrameRate_Num>24000</FrameRate_Num>
<FrameRate_Den>1001</FrameRate_Den>
<StreamSize>194113</StreamSize>
<StreamSize_String>190 KiB (100%)</StreamSize_String>
<StreamSize_String1>190 KiB</StreamSize_String1>
<StreamSize_String2>190 KiB</StreamSize_String2>
<StreamSize_String3>190 KiB</StreamSize_String3>
<StreamSize_String4>189.6 KiB</StreamSize_String4>
<StreamSize_String5>190 KiB (100%)</StreamSize_String5>
<StreamSize_Proportion>1.00000</StreamSize_Proportion>
<File_Modified_Date>2024-08-09 03:19:16 UTC</File_Modified_Date>
<File_Modified_Date_Local>2024-08-08 23:19:16</File_Modified_Date_Local>
<extra>
<FileExtension_Invalid>dfxp ttml</FileExtension_Invalid>
</extra>
</track>
<track type="Text">
<Count>305</Count>
<StreamCount>1</StreamCount>
<StreamKind>Text</StreamKind>
<StreamKind_String>Text</StreamKind_String>
<StreamKindID>0</StreamKindID>
<Format>TTML</Format>
<Format_String>TTML</Format_String>
<Format_Info>Timed Text Markup Language</Format_Info>
<Format_Url>https://en.wikipedia.org/wiki/Timed_Text_Markup_Language</Format_Url>
<Format_Commercial>TTML</Format_Commercial>
<Format_Profile>SMPTE-TT</Format_Profile>
<Duration>6424.383</Duration>
<Duration_String>1 h 47 min</Duration_String>
<Duration_String1>1 h 47 min 4 s 383 ms</Duration_String1>
<Duration_String2>1 h 47 min</Duration_String2>
<Duration_String3>01:47:04.383</Duration_String3>
<Duration_String4>01:46:57:23</Duration_String4>
<Duration_String5>01:47:04.383 (01:46:57:23)</Duration_String5>
<Duration_Start>3605.856</Duration_Start>
<Duration_Start_String>1 h 0 min</Duration_Start_String>
<Duration_Start_String1>1 h 0 min 5 s 856 ms</Duration_Start_String1>
<Duration_Start_String2>1 h 0 min</Duration_Start_String2>
<Duration_Start_String3>01:00:05.856</Duration_Start_String3>
<Duration_Start_String4>01:00:02:06</Duration_Start_String4>
<Duration_Start_String5>01:00:05.856 (01:00:02:06)</Duration_Start_String5>
<Duration_End>10030.239</Duration_End>
<Duration_End_String>2 h 47 min</Duration_End_String>
<Duration_End_String1>2 h 47 min 10 s 239 ms</Duration_End_String1>
<Duration_End_String2>2 h 47 min</Duration_End_String2>
<Duration_End_String3>02:47:10.239</Duration_End_String3>
<Duration_End_String4>02:47:00:05</Duration_End_String4>
<Duration_End_String5>02:47:10.239 (02:47:00:05)</Duration_End_String5>
<Duration_Base>media</Duration_Base>
<FrameRate_Mode>CFR</FrameRate_Mode>
<FrameRate>23.976</FrameRate>
<FrameRate_String>23.976 (24000/1001) FPS</FrameRate_String>
<FrameRate_Num>24000</FrameRate_Num>
<FrameRate_Den>1001</FrameRate_Den>
<FrameCount>154031</FrameCount>
<Compression_Mode>Lossless</Compression_Mode>
<Compression_Mode_String>Lossless</Compression_Mode_String>
<Language>es-419</Language>
<Language_String>Spanish (Latin America)</Language_String>
<Language_String1>Spanish (Latin America)</Language_String1>
<Language_String2>es</Language_String2>
<Language_String3>spa</Language_String3>
<Language_String4>es-419</Language_String4>
<Events_Total>1380</Events_Total>
<Lines_Count>2077</Lines_Count>
<Lines_MaxCountPerEvent>2</Lines_MaxCountPerEvent>
</track>
</media>
</MediaInfo>
JeromeMartinez commented 3 months ago

TTML related outputs already shows TimeCode_Start & TimeCode_End, but only if there is a time code rather that a time stamp in the file. The difference is that TimeCode_Start & TimeCode_End are not guaranteed to be coherent compared to the time line as a time code may have a discontinuity. It was initially designed for dedicated time code tag in a video stream or a dedicated time code track linked to a video stream, with a difference between the value of the time code and the time stamp of this time code. This is the reason that TimeCode_End is not the time code after the end of the last frame (it is a value which can have a discontinuity after the time code of the last frame), it is the time code value linked to the last frame.

We are using code to generate the the timecode policy to determine if the text is within the in/out of the composition.

My understanding here is that you want info about a time line, with the info displayed in a time code format. TimeCode_Start was not created with that in mind as it is linked to formats potentially having discontinuities, so time line is not the time code sequence (duration is not end time code minus start time code). Duration_Start is for the start of the stream, Duration_Start_Command is for the start of the commands, and TimeCode_Start is for time code values not always in sync with the timeline, it seems that I need a dedicated field that shows something like "Command" but for display. Something like TimeCode_Start_Display. As it is a timeline thing, TimeCode_End_Display would be the time (and the conversion to to time code) of when the display disappears (so theoretical converted time code of the frame after the last displayed one).

Not difficult, we already have all internally, the issue here is how to represent different ideas (time code as a value able to have discontinuities, or as a representation of a time), and I need also to clarify the documentation about the meaning of each field.

joe-sciame-wm commented 3 months ago

Jerome,

Thanks for the explanation. Yes, here is our use case. We have a composition timeline that says, for example, that the SOM is 01:00:00:00 and the EOM is 01:00:10:00. We want to ensure all of the timed text that is "content" is within the the composition as part of a sniff check. This allows us to at a glance understand that the timed text files are within the composition and we can have an operations team look a it if there is content outside. Having a consistent field across all text formats would help with this. The explanation about the "only there if there is a timecode" makes sense as I did see it in TTML's at one point but it must have been those with HH:MM:SS:FF. Event those values in TTML are not timecodes but a mediatime representation in timecode format so it has to be converted from mediatime to timecode to get actual timecode location.

JeromeMartinez commented 3 months ago

We use to focus on start and end of a stream iven if there is no content, it makes sense that you want something more related to the display.

That said, wouldn't it preferable to use the Duration_Start_Command? If the start of commands is outside of the video time-line it may be deleted during a conversion and the caption will not be displayed correctly. Same output for TTML (no command specific stuff), not for SCC (608 needs some previous frames).

If displayed time stamp / code is preferred over commands time stamp / code, only a small couple of hours of work, I'll implement that after my vacation.

joe-sciame-wm commented 3 months ago

Hi Jerome, I agree with you that what we would want is the Duration Start and End Command so that as you say nothing gets cut off. On TTML though I'm not seeing this field in the XML. This is a common problem we've seen with SCC.

<Format_Profile>EBU-TT / IMSC1</Format_Profile>
<Duration>123.206</Duration>
<Duration_String>2 min 3 s</Duration_String>
<Duration_String1>2 min 3 s 206 ms</Duration_String1>
<Duration_String2>2 min 3 s</Duration_String2>
<Duration_String3>00:02:03.206</Duration_String3>
<Duration_String4>00:02:03:02</Duration_String4>
<Duration_String5>00:02:03.206 (00:02:03:02)</Duration_String5>
<Duration_Start>3.378</Duration_Start>
<Duration_Start_String>3 s 378 ms</Duration_Start_String>
<Duration_Start_String1>3 s 378 ms</Duration_Start_String1>
<Duration_Start_String2>3 s 378 ms</Duration_Start_String2>
<Duration_Start_String3>00:00:03.378</Duration_Start_String3>
<Duration_Start_String4>00:00:03:09</Duration_Start_String4>
<Duration_Start_String5>00:00:03.378 (00:00:03:09)</Duration_Start_String5>
<Duration_End>126.584</Duration_End>
<Duration_End_String>2 min 6 s</Duration_End_String>
<Duration_End_String1>2 min 6 s 584 ms</Duration_End_String1>
<Duration_End_String2>2 min 6 s</Duration_End_String2>
<Duration_End_String3>00:02:06.584</Duration_End_String3>
<Duration_End_String4>00:02:06:11</Duration_End_String4>
<Duration_End_String5>00:02:06.584 (00:02:06:11)</Duration_End_String5>
joe-sciame-wm commented 2 months ago

Hi Jerome,

Can I have an update on this one?

JeromeMartinez commented 2 months ago

It is in the latest snapshots, please confirm that it fits your needs. Note that I have set the fields hidden by default in the CLI/GUI because it is redundant and it becomes too many fields for text, but it is still available in the full text view and in XML or JSON.