Relationship of Stored, Sampled and Display size in modern infrastructure

[gzander]

What is the intention of Stored, Sampled and Display Size in a modern infrastructure for compressed video?

[SMPTE-standards]

Historically they all applied to uncompressed images. When compression was introduced, different mappings have probably done slightly different things. We need to audit this as part of the 377-1 revision.

My opinion is:

• Display Rectangle is the cropped output of a theoretical MXF player
• Sampled Rectangle includes all the active pixels that go into and come out of a compression encoder / decoder. For example putting a pixel matrix of 1920 x 1080 into an encoder and expecting the same pixel matrix to come out again.
• Stored Rectangle is the pixel matrix stored by the codec in order to contain the sampled Rectangle. It is codec dependant and theoretical - you con't really get at it. For example 1920 x 1088

As we revise MXF, I think Display and Sampled Rectangles should be required and the status Stored Rectangle is codec dependent and probably optional for everything except Uncompressed.

[mrmxf]

Further information from an email thread discussing whether DisplayF2Offset or SampledF2Offset should be used to indicate Topness:

The most common case nowadays is when Stored Rectangle = Sampled Rectangle and (Sampled Y Offset == Displayed Y Offset == 0 ). I think this is the case for all ProRes files. In compression system, I have viewed the Stored and Sample rectangles as internal to the compression engine and thus are nearly always the same thing Any attempt to make them different has traditionally caused confusion and interop issues (e.g. rectangle heights of 1088 or 1080). The safest thing for consistent behaviour, I feel, is to use Display F2 Offset. This will give consistent results if the Display rectangle is changed. Using StoredF2 Offset creates an ambiguous corner case when the Display rectangle is used to crop an interlaced source resulting in both DisplayF2Offset and StoredF2Offset being -1. Although Luis captured this in his spreadsheet, I don't think that the standard unambiguously supports that interpretation.

Bottom line - use DisplayF2Offset. The standard is ambiguous in best practise, but not in interprestation.

[thomasheritage]

Note that https://github.com/SMPTE/st2067-2/issues/54#issuecomment-732334805 explains how Display Rectangle and Production Aperture have been considered equivalent by MXF implementers (RP 187 describes the Production Aperture as the "maximum possible image area that can be supported by the video standard").

So, I think it's the case that:

• You would expect the Display Rectangle to always be the same as the Sampled Rectangle for a modern standard like UHD; but the Display Rectangle could be a few lines smaller than the Sampled Rectangle for something like SD (if a few lines of blanking (VBI) have been encoded).
• You wouldn't use the Display Rectangle as a mechanism to arbitrarily crop an image (for example to remove black bars from "letterboxed" content).

[palemieux]

You wouldn't use the Display Rectangle as a mechanism to arbitrarily crop an image (for example to remove black bars from "letterboxed" content).

Definitely. IMF introduces an explicit Active Area Rectangle for that use case: Annex G at https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9097478