search

Netflix / photon

Photon is a Java implementation of the Interoperable Master Format (IMF) standard. IMF is a SMPTE standard whose core constraints are defined in the specification st2067-2:2013
Apache License 2.0
234 stars 76 forks source link

Invalid marker offset? #259

Open wabiloo opened 4 years ago

wabiloo commented 4 years ago

I am trying to parse and validate an IMP for a 5 minute asset. The CPL looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<CompositionPlaylist xmlns="http://www.smpte-ra.org/schemas/2067-3/2016" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Id>urn:uuid:7b38e773-9922-49ae-85d1-03a68fd6a701</Id>
  <Annotation>Ownzones</Annotation>
  <IssueDate>2019-06-26T11:52:49.080Z</IssueDate>
  <Issuer>Ownzones</Issuer>
  <Creator>Ownzones Connect Ver. release 65 22435</Creator>
  <ContentOriginator>Ownzones</ContentOriginator>
  <ContentTitle>20519845</ContentTitle>
  <EssenceDescriptorList>
    <EssenceDescriptor xmlns:r0="http://www.smpte-ra.org/reg/395/2014/13/1/aaf" xmlns:r1="http://www.smpte-ra.org/reg/335/2012" xmlns:r2="http://www.smpte-ra.org/reg/2003/2012">
      <Id>urn:uuid:af0f8f88-d0cc-4d4b-a9c9-cbb5fe2b277a</Id>
       ...
    </EssenceDescriptor>
    <EssenceDescriptor xmlns:r0="http://www.smpte-ra.org/reg/395/2014/13/1/aaf" xmlns:r1="http://www.smpte-ra.org/reg/335/2012" xmlns:r2="http://www.smpte-ra.org/reg/2003/2012">
      <Id>urn:uuid:52bfb7ed-8e19-4fdd-89f6-90bb89ae5b11</Id>
      ...
    </EssenceDescriptor>
  <CompositionTimecode>
    <TimecodeDropFrame>false</TimecodeDropFrame>
    <TimecodeRate>25</TimecodeRate>
    <TimecodeStartAddress>00:00:00:00</TimecodeStartAddress>
  </CompositionTimecode>
  <EditRate>25 1</EditRate>
  <ExtensionProperties>
    <cc:ApplicationIdentification xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016">http://www.smpte-ra.org/schemas/2067-21/2016</cc:ApplicationIdentification>
  </ExtensionProperties>
  <SegmentList>
    <Segment>
      <Id>urn:uuid:00292f02-3fa4-46c6-ab39-d58d5e2db4fc</Id>
      <Annotation>segment</Annotation>
      <SequenceList>
        <cc:MarkerSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-3/2016">
          <Id>urn:uuid:4e09c29a-0276-48cc-9839-b75d1001c0d8</Id>
          <TrackId>urn:uuid:ffa2793c-3b59-4c54-89ba-597410866e38</TrackId>
          <ResourceList>
            <Resource xsi:type="MarkerResourceType">
              <Id>urn:uuid:3fa82cd6-220d-4e86-afc6-4b8042fb1392</Id>
              <Annotation>Resource</Annotation>
              <IntrinsicDuration>7500</IntrinsicDuration>
              <Marker>
                <Label>FFOC</Label>
                <Offset>0</Offset>
              </Marker>
              <Marker>
                <Label>LFOC</Label>
                <Offset>7500</Offset>
              </Marker>
            </Resource>
          </ResourceList>
        </cc:MarkerSequence>
        <cc:MainImageSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016">
          <Id>urn:uuid:3a50a999-5438-4bd0-9648-64639ba06050</Id>
          <TrackId>urn:uuid:635136aa-2191-4765-aed8-c444f44fd610</TrackId>
          <ResourceList>
            <Resource xsi:type="TrackFileResourceType">
              <Id>urn:uuid:528c2535-89f1-46a2-b6d5-afc5dc222bfa</Id>
              <Annotation>Resource</Annotation>
              <EditRate>25 1</EditRate>
              <IntrinsicDuration>7500</IntrinsicDuration>
              <EntryPoint>0</EntryPoint>
              <SourceDuration>7500</SourceDuration>
              <SourceEncoding>urn:uuid:af0f8f88-d0cc-4d4b-a9c9-cbb5fe2b277a</SourceEncoding>
              <TrackFileId>urn:uuid:3039fb60-6954-4958-944d-0a90357e193f</TrackFileId>
            </Resource>
          </ResourceList>
        </cc:MainImageSequence>
        <cc:MainAudioSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016">
          <Id>urn:uuid:6cfe498d-ab02-4642-a09a-f2ff48fcc67b</Id>
          <TrackId>urn:uuid:8a0e2bfa-4dd8-489e-a51c-1dcce1520cec</TrackId>
          <ResourceList>
            <Resource xsi:type="TrackFileResourceType">
              <Id>urn:uuid:983441b8-839b-4518-988e-614125e4abc0</Id>
              <Annotation>Resource</Annotation>
              <EditRate>48000 1</EditRate>
              <IntrinsicDuration>14400000</IntrinsicDuration>
              <EntryPoint>0</EntryPoint>
              <SourceDuration>14400000</SourceDuration>
              <SourceEncoding>urn:uuid:52bfb7ed-8e19-4fdd-89f6-90bb89ae5b11</SourceEncoding>
              <TrackFileId>urn:uuid:b190b32f-b07c-4247-b917-dcf990010f2f</TrackFileId>
            </Resource>
          </ResourceList>
        </cc:MainAudioSequence>
      </SequenceList>
    </Segment>
  </SegmentList>
</CompositionPlaylist>

On parsing, I get the following error: ERROR-VirtualTrack with ID ffa2793c-3b59-4c54-89ba-597410866e38 has a resource with ID urn:uuid:3fa82cd6-220d-4e86-afc6-4b8042fb1392, that has a marker LFOC, that has an invalid offset value 7500, should be in the range [0,7499] [Photon version: 4.5.14]

This error is thrown in the IMFCoreConstraintsChecker.checkVirtualTrackResourceList, (https://github.com/Netflix/photon/blob/master/src/main/java/com/netflix/imflibrary/st2067_2/IMFCoreConstraintsChecker.java#L357).

I don't have a version of the st2067-3:2013 spec, but I do have st2067-3:2016 which I believe supersedes it, and i don't find anything in there that prevents the offset value to be the same as the intrinsic duration. Is this because when looking at offsets they should be considered as 0-indexed, whereas duration is 1-indexed? Again, I don't see anything in the spec docs I have about this... Or is this check too constraining (and that test should use '>' instead of '>-' when comparing instrinsic duration and marker offset)?

davidt-netflix commented 4 years ago

I believe you have correctly identified the cause. Duration begins counting from 1, while Offset (and EntryPoint) begins counting from 0.

According to SMPTE 2067-3:2013 6.14.3

The Offset element shall define the position of the marker from the start of the Marker Resource timeline. It shall be expressed in Resource Edit Units.

And in 6.14

The native start point of a MarkerResourceType instance shall be the start of the timeline, i.e. correspond to an offset of 0 on the timeline.

That means an offset of zero would be the first edit unit in the Marker Resource timeline. An offset of 7500 would be the edit unit immediately following the last edit unit (7499) in the Marker Resource timeline.

However, there is a bit of ambiguity from another part of section 6.14

The native duration of a MarkerResourceType instance, as indicated by the IntrinsicDuration element, shall be set to any value equal or larger to the largest Offset value within all its Marker elements. (emphasis mine)

I suspect that this may be a mistake, and the intended meaning was probably any value larger. Otherwise, there will be off-by-one issues with any marker on the last edit unit.

Regardless, in this specific specific case, I believe that Photon correctly identified the error. The marker LFOC identifies a specific video frame. Video frame EntryPoints are definitely zero based, which means that the last frame has an entry point of 7499, and there is no frame 7500.

In my opinion, this validation shouldn't be changed, although we should look for clarification of the wording in 6.14. I'd welcome other opinions as well though.