add block error for truncated blocks

[dericed]

This is to supersede the attempted method in https://github.com/MediaArea/MediaConch/pull/136. To recreate:

ffmpeg -f lavfi -i testsrc -c:v v210 -vframes 1 test.mkv
head -c 200000 test.mkv > broken.mkv
mediaconch -fx -mt -mi broken.mkv 

broken.mkv should end with a truncated Cluster. broken.mkv passes mediaconch when it has an obvious problem.

Last Cluster of mediatrace is:

<block offset="716" name="Cluster" size="25">
    <block offset="716" name="Header" size="12">
      <data offset="716" name="Name">256095861</data>
      <data offset="720" name="Size">215057</data>
    </block>
    <block offset="728" name="CRC-32" size="6">
      <block offset="728" name="Header" size="2">
        <data offset="728" name="Name">63</data>
        <data offset="729" name="Size">4</data>
      </block>
      <data offset="730" name="Value" info="Not tested 2 2261744067">2261744067</data>
    </block>
    <block offset="734" name="Timecode" info="0" size="3">
      <block offset="734" name="Header" size="2">
        <data offset="734" name="Name">103</data>
        <data offset="735" name="Size">1</data>
      </block>
      <data offset="736" name="Data">0</data>
    </block>
  </block>
</block>

I suggest that the Cluster contain an error (also the parent elements would likely get the same error), so new output should look like:

<block offset="716" name="Cluster" error="MKV-TRUNCATED-ELEMENT:1" size="25">
    <block offset="716" name="Header" size="12">
      <data offset="716" name="Name">256095861</data>
      <data offset="720" name="Size">215057</data>
    </block>
    <block offset="728" name="CRC-32" size="6">
      <block offset="728" name="Header" size="2">
        <data offset="728" name="Name">63</data>
        <data offset="729" name="Size">4</data>
      </block>
      <data offset="730" name="Value" info="Not tested 2 2261744067">2261744067</data>
    </block>
    <block offset="734" name="Timecode" info="0" size="3">
      <block offset="734" name="Header" size="2">
        <data offset="734" name="Name">103</data>
        <data offset="735" name="Size">1</data>
      </block>
      <data offset="736" name="Data">0</data>
    </block>
  </block>
</block>

If the approached used in transferrable to other element-based formats (QuickTime, AVI), then I suggest to use a more general error such as "TRUNCATED-ELEMENT:1" rather than "MKV-TRUNCATED-ELEMENT:1"

[JeromeMartinez]

there is already a "IsTruncated" field in MediaInfo report for that, anyway it may be interesting to add it in the trace too. I currently don't catch truncated sub elements, I'll first add it at the Segment level.

[JeromeMartinez]

was actually easier than expected to have on all elements:

<block offset="47" name="Segment" size="753">
    <block offset="47" name="Header" size="12">
        <data offset="47" name="Name">139690087</data>
        <data offset="51" name="Size" error="TRUNCATED-ELEMENT:1">742</data>
    </block>
    <block>[...]</block>
    <block offset="772" name="Cues" size="28">
        <block offset="772" name="Header" size="12">
            <data offset="772" name="Name">206814059</data>
            <data offset="776" name="Size" error="TRUNCATED-ELEMENT:1">17</data>
        </block>
        <block offset="784" name="CuePoint" size="16">
            <block offset="784" name="Header" size="2">
                <data offset="784" name="Name">59</data>
                <data offset="785" name="Size" error="TRUNCATED-ELEMENT:1">15</data>
            </block>
            <block offset="786" name="CueTime" info="0" size="3">
                <block offset="786" name="Header" size="2">
                    <data offset="786" name="Name">51</data>
                    <data offset="787" name="Size">1</data>
                </block>
                <data offset="788" name="Data">0</data>
            </block>
            <block offset="789" name="CueTrackPositions" size="11">
                <block offset="789" name="Header" size="2">
                    <data offset="789" name="Name">55</data>
                    <data offset="790" name="Size" error="TRUNCATED-ELEMENT:1">10</data>
                </block>
                <block offset="791" name="CueTrack" info="1" size="3">
                    <block offset="791" name="Header" size="2">
                        <data offset="791" name="Name">119</data>
                        <data offset="792" name="Size">1</data>
                    </block>
                    <data offset="793" name="Data">1</data>
                </block>
                <block offset="794" name="CueClusterPosition" info="680" size="4">
                    <block offset="794" name="Header" size="2">
                        <data offset="794" name="Name">113</data>
                        <data offset="795" name="Size">2</data>
                    </block>
                    <data offset="796" name="Data">680</data>
                </block>
                <block offset="798" name="CueRelativePosition" size="2">
                    <block offset="798" name="Header" size="2">
                        <data offset="798" name="Name">112</data>
                        <data offset="799" name="Size" error="TRUNCATED-ELEMENT:1">1</data>
                    </block>
                </block>
            </block>
        </block>
    </block>
</block>

Is it OK?

Note: the test file is not good for unit testing because there are more errors due to your hard truncation. We need to find a way to create a file with only this error.

[dericed]

I am curious about the position of the error. It is on the Element Data Size even when it seems like it is not the VINT of the Element Data Size which is truncated. Why not include this on the Element's block?

[JeromeMartinez]

Good argument. I as focusing on the position I test, the impact.

But... with error at the element's block, I am afraid about when we'll have several issues related to the Element's block for a similar argument. How will we do? "e1", "e2"... attributes? ;-)

[dericed]

I prefer a delimiter rather than e1 e2. Semicolon? e="ERRORNAME:ERRORVERSION;ERRORNAME:ERRORVERSION"

[JeromeMartinez]

More difficult on my side, but not so much. fine. but in that case XSL must be ready for that.

[dericed]

Other alternative is:

<block>
  <error/>
  <error/>
</block>

but this method doesn't work for <data>. I suggest e attribute with delimiter is the best option.

[JeromeMartinez]

let's stay with e