exponential-decay / skeleton-test-suite-generator

DROID Skeleton Test Suite Generator (skeleton-test-suite-generator): Tool for the automated generation of digital objects based on the digital signatures documented in the PRONOM database maintained by The National Archives, UK. The skeleton-test-suite-generator serves to fill the gap that exists whereby the community requires a corpus of digital objects for the validation and evaluation of format identification tools and techniques. The tool should be used to complement a methodology whereby skeleton files are also generated manually by signature developers. The tool takes a signature specified for a digital object in PRONOM and constructs a digital object that will match its footprint. For more information, see the README.md associated with the project...
zlib License
8 stars 2 forks source link

Out of order EOF sequences #3

Closed richardlehane closed 7 years ago

richardlehane commented 10 years ago

Hi Ross This new signature from the latest PRONOM batch doesn't seem to generate a proper test: http://apps.nationalarchives.gov.uk/PRONOM/fmt/648

The issue is with this bit:

Position type Absolute from EOF Offset 8 Value 30333069

which isn't appearing in an absolute position from the EOF.

This seems a bit of a dodgy one anyway as the TNA's example "BOF: 030i (OFFSET 4) EOF: Svar (Offset 62) EOF: 030i (OFFSET 8)" doesn't conform to the signature because that "Svar" sequence should be between offset 20-46 from the EOF, not offset 62.

thanks! Richard

ross-spencer commented 10 years ago

Hi Richard,

Yep, that's broken. I'm pretty sure I know what's going on / what I'm not doing in the code so can get that fixed. Will just need a little time.

Have you managed to create a sample file manually? - I have but DROID doesn't like it, so I'm wondering if I'm struggling with the offset logic this morning. I'll see if I can wrangle a sample from TNA and point them in the direction of their description issue too.

Many thanks,

Ross

ross-spencer commented 10 years ago

File we're struggling with:

https://gist.github.com/ross-spencer/a289b486851ef7fc6f51

  <?xml version="1.0" encoding="utf-8"?>
  <PRONOM-Report xmlns="http://pronom.nationalarchives.gov.uk">
    <report_format_detail>
      <FileFormat>
        <FormatID>1447</FormatID>
        <FormatName>Media View Pro</FormatName>
        <FormatVersion>
        </FormatVersion>
        <FormatAliases>
        </FormatAliases>
        <FormatFamilies>
        </FormatFamilies>
        <FormatTypes>Presentation</FormatTypes>
        <FormatDisclosure>
        </FormatDisclosure>
        <FormatDescription>Proprietary cataloguing program to manage photo and video assets.</FormatDescription>
        <BinaryFileFormat>
        </BinaryFileFormat>
        <ByteOrders>
        </ByteOrders>
        <ReleaseDate>
        </ReleaseDate>
        <WithdrawnDate>
        </WithdrawnDate>
        <ProvenanceSourceID>1</ProvenanceSourceID>
        <ProvenanceName>Digital Preservation Department / The National Archives</ProvenanceName>
        <ProvenanceSourceDate>14 Jul 2014</ProvenanceSourceDate>
        <ProvenanceDescription>
        </ProvenanceDescription>
        <LastUpdatedDate>14 Jul 2014</LastUpdatedDate>
        <FormatNote>
        </FormatNote>
        <FormatRisk>
        </FormatRisk>
        <TechnicalEnvironment>
        </TechnicalEnvironment>
        <FileFormatIdentifier>
          <Identifier>fmt/648</Identifier>
          <IdentifierType>PUID</IdentifierType>
        </FileFormatIdentifier>
        <ExternalSignature>
          <ExternalSignatureID>1497</ExternalSignatureID>
          <Signature>mpcatalog</Signature>
          <SignatureType>File extension</SignatureType>
        </ExternalSignature>
        <InternalSignature>
          <SignatureID>976</SignatureID>
          <SignatureName>Media View Pro</SignatureName>
          <SignatureNote>BOF: 030i (OFFSET 4)    EOF: Svar (Offset 62)    EOF: 030i (OFFSET 8)</SignatureNote>
          <ByteSequence>
            <ByteSequenceID>1202</ByteSequenceID>
            <PositionType>Absolute from BOF</PositionType>
            <Offset>4</Offset>
            <MaxOffset>
            </MaxOffset>
            <IndirectOffsetLocation>
            </IndirectOffsetLocation>
            <IndirectOffsetLength>
            </IndirectOffsetLength>
            <Endianness>
            </Endianness>
            <ByteSequenceValue>30333069</ByteSequenceValue>
          </ByteSequence>
          <ByteSequence>
            <ByteSequenceID>1203</ByteSequenceID>
            <PositionType>Absolute from EOF</PositionType>
            <Offset>8</Offset>
            <MaxOffset>
            </MaxOffset>
            <IndirectOffsetLocation>
            </IndirectOffsetLocation>
            <IndirectOffsetLength>
            </IndirectOffsetLength>
            <Endianness>
            </Endianness>
            <ByteSequenceValue>30333069</ByteSequenceValue>
          </ByteSequence>
          <ByteSequence>
            <ByteSequenceID>1204</ByteSequenceID>
            <PositionType>Absolute from EOF</PositionType>
            <Offset>20</Offset>
            <MaxOffset>46</MaxOffset>
            <IndirectOffsetLocation>
            </IndirectOffsetLocation>
            <IndirectOffsetLength>
            </IndirectOffsetLength>
            <Endianness>
            </Endianness>
            <ByteSequenceValue>53566172</ByteSequenceValue>
          </ByteSequence>
        </InternalSignature>
        <RelatedFormat>
          <RelationshipType>Is subsequent version of</RelationshipType>
          <RelatedFormatID>1446</RelatedFormatID>
          <RelatedFormatName>Microsoft Expression Media</RelatedFormatName>
          <RelatedFormatVersion>2</RelatedFormatVersion>
        </RelatedFormat>
      </FileFormat>
      <SearchCriteria>Criteria</SearchCriteria>
    </report_format_detail>
  </PRONOM-Report>
ross-spencer commented 9 years ago

Fixed in DROID signature file v79. Leave as open for now as this might still crop up when creating skeleton files.