Test for maximum frame and frame color component size

[matmat]

Please see the following references:

ISO 15444-1:2006 Amendment 1

A.10.1 Codestream restrictions for digital cinema

(...)

2K digital cinema profile	4K digital cinema profile
Application specific restrictions
Max compressed bytes for any image frame (aggregate of all 3 color components)	1302083 bytes for 24 fps 651041 bytes for 48 fps	1302083 bytes (for 24 fps)
Max compressed bytes for any single color component of an image frame	1041666 bytes for 24 fps 520833 bytes for 48 fps	1041666 bytes for 2K portion of each component (for 24 fps)

DCSS v1.3 ( https://www.dcimovies.com/specification/DCI_DCSS_Ver1-3_2018-0627.pdf )

4.3.3. Codestream Specification All codestreams shall fully conform with [ISO 15444-1:2006 Amendment 1], as more fully constrained as follows:

(...)

• For a frame rate of 24 FPS, a 2K distribution shall have a maximum of 1,302,083 bytes per frame (aggregate of all three color components including headers). Additionally, it shall have a maximum of 1,041,666 bytes per color component per frame including all relevant tile-part headers
• For a frame rate of 48 FPS, a 2K distribution shall have a maximum of 651,041 bytes per frame (aggregate of all three color components including headers). Additionally, it shall have a maximum of 520,833 bytes per color component per frame including all relevant tile-part headers.
• A 4K distribution shall have a maximum of 1,302,083 bytes per frame (aggregate of all three color components including headers). Additionally, the 2K portion of each frame shall satisfy the 24 FPS 2K distribution requirements as stated above.

Note: For information purposes only, this yields a maximum of 250 Mbits/sec total and a maximum of 200 Mbits/sec for the 2K portion of each color component.

CTP v1.2 ( https://www.dcimovies.com/compliance_test_plan/DCI_CTP_v12.pdf )

4.5.2. Image Compression Standard & Encoding Parameters

(...)

• Verify that every frame is correctly JPEG 2000 encoded as described in [ISO-15444-1]. Verify that the proper JPEG 2000 encoding parameters as specified in [ISO-15444-1-AMD-1] were used. The Codestream Specifications for 2K and 4K distributions are listed in [DCI-DCSS-1-2], section [4.3]. This can be achieved by using JPEG 2000 decoding software. (...) If any frame fails to correctly decode or does not conform to the appropriate Codestream Specifications, this is cause to fail the test.

(...)

C.5.1. j2c-scan Source Code Listing

SMPTE ST 429-2:2013

10.2.2 Compression Picture essence shall be compressed using JPEG 2000 [ISO/IEC 15444-1] as constrained by [ISO/IEC 15444-1:2004/Amd 1:2006].

SMPTE ST 429-4:2006

5 JPEG 2000 Codestream Restrictions The JPEG 2000 codestreams shall be restricted according to ISO-IEC 15444-1 Amd 1. This amendment defines two profiles for D-Cinema usage referred to in this standard as 2K and 4K.

P-HFR -- Prototype DCP for High Frame Rate Projection ( https://isdcf.com/papers/ISDCF-HighFrameRate-DCP.pdf )

2.1 JPEG2000 Codestream Formulation JPEG 2000 Codestreams used in P-HFR files shall conform to ISO/IEC 15444-1:2004, “JPEG 2000 Image Coding System: Core Coding System” and ISO/IEC 15444-1:2004/Amd 1:2006, Profiles for Digital Cinema Applications” except that the constraints on bit rate shall be expanded as follows:

• The data rate of the total codestream shall not exceed 500Mb/s
• The combined data rates of the Cx and Cz channels shall not exceed 250Mb/s
• The data rate of the Y channel alone shall not exceed 500Mb/s

(...)

2.3 Picture Track File Formulation A stereoscopic Picture Track File in a P-HFR composition shall conform to SMPTE ST 429-13:2009 with the following exceptions:

• The file shall contain only 2k images.
• In the MXF FileDescriptor class, the SampleRate property shall be set to indicate the total frame rate of the file. For stereoscopic content this value shall be twice the EditRate (i.e., one of 96/1, 100/1 or 120/1).

(...)

2.4 Composition Formulation (...) All files in a P-HFR composition shall have equal edit rate, one of 48/1, 50/1 or 60/1.

[matmat]

The above means that there are really no constraints on actual bitrate. Bitrate constraint is really just a consequence of two constraints on each frame:

1. Max compressed bytes for any single color component of an image frame
2. Max compressed bytes for any single (or specific, for HFR) color component of an image frame

From these two variable you could derive a maximum bitrate, but this is not the limiting factor (according to the docs at least)

This is important, because a single frame out of spec can make the composition unplayable on some hardware. This is the result when ingesting such a DCP on a Dolby DSS200 with Cat745 mediablock with latest software, v4.9.4(5), from the logs:

_2019-11-19 16:27:50,922 WARN [Thread-3458927] mxf.DecoderVideoSupport - Unsupported MXF file: Picture essence contains (at least) one frame with an unsupported frame size of 2,004,861 bytes. Picture essence metadata: (File: 89170ba1-a554-4773-b303-8df6ae85fd04j2c.mxf, Type: JPEG, Frame Rate: 24000/1000, Sample Rate: 24/1, Dimensions: 2D, Max Frame Size: 2004861, Max Bit Rate: 381694928/1, Average Bit Rate: 3286357/1, Duration: 1768.2916)

In this case both the average and max bitrate is well within the capacity of the Cat745. It is enough for one single frame out of spec to make it unplayable!

[matmat]

After som light reverse-engineering, I was able to find the specifications for Dolby mediablocks (those made before they bought Doremi):

DSP100_FRAME_BUFFER_SIZE = 2097152;
CAT862_FRAME_BUFFER_SIZE = 2097152;
CAT745_FRAME_BUFFER_SIZE = 1572864;
DSP100_MAX_BIT_RATE_JPEG = 350000000;
DSP100_MAX_BIT_RATE_MPEG = 140000000;
CAT862_MAX_BIT_RATE_JPEG = 350000000;
CAT862_MAX_BIT_RATE_MPEG = 140000000;
CAT745_MAX_BIT_RATE_JPEG = 550000000;
CAT745_MAX_BIT_RATE_MPEG = 300000000;

Sizes in bytes.

It is interesting to note that the older mediablocks (DSP100 and Cat862) have an higher per-frame buffer than the newer Cat745 mediablock. Though the Cat745 has a higher average bit rate capacity.

And indeed, I have tested the above DCP, and it plays totally fine on a Cat862! But not on the newer (and 4K capable) Cat745.

[matmat]

I just discovered that the ISO/IEC 15444-1 has been revised, the current version is ISO/IEC 15444-1:2019 ( https://www.iso.org/standard/78321.html ). The contents appears to be the same as ITU-T Rec. T.800 which is freely available.

Here the writing has shifted to specify the bitrate first, and then derive the maximum frame and component sizes.

SMPTE ST 429-2:2013 has been amended to reference the next newest version of 15444-1, but the current versions of the DCSS, CTP, 429-4 and ISDCF's HFR spec all reference the old, 15444-1:2004/Amd 1:2006. The 429-2 ammendment also contains additional frame rates.

SMPTE ST429-2.2013Am1.2018

1. Updates to section 3 “Normative References”.

(...)

• The following reference: “[ISO/IEC 15444-1:2004] Information Technology — JPEG 2000 Image Coding System: Core Coding System”
  shall be changed into:
  “[ISO/IEC 15444-1:2016] Information Technology — JPEG 2000 Image Coding System: Core Coding System”

• The reference to “[ISO/IEC 15444-1:2004/Amd 1:2006] Profiles for Digital Cinema Applications” shall be removed entirely.

(...)

2. Update section 8.1 - “Edit Rate”

The line

“The composition shall have an Edit Rate of 24/1 or 48/1. (See also Annex B.)”

shall be changed Into

“The composition shall have an Edit Rate of 24/1, 25/1, 30/1, 48/1, 50/1 or 60/1.”

(...)

5 - Update version of JPEG reference in section 10.2.2 - “Compression”

Change the line

“Picture essence shall be compressed using JPEG 2000 [ISO/IEC 15444-1] as constrained by [ISO/IEC 15444-1:2004/Amd 1:2006].”

Into

“Picture essence shall be compressed using JPEG 2000 [ISO/IEC 15444-1].”

TU-T Rec. T.800 (06/2019) ( https://www.itu.int/rec/T-REC-T.800/en )

Table A.46 – Codestream restrictions for digital cinema applications

2K digital cinema profile	4K digital cinema profile
Maximum instantaneous bitrate for all 3 colour components2	Including bytes of possible PLT markers 250×106 Bit/s	Including bytes of possible PLT markers 250×106 Bit/s
Maximum instantaneous bitrate for each single colour component including all relevant tile-part headers.	Including bytes of possible PLT markers 200×106 Bit/s	Including bytes of possible PLT markers 200×106 Bit/s for 2K portion of each component
Maximum frame rate	60	30

² The maximum instantaneous bit-rate must not be exceeded. The maximum admissible compressed bytes are explained in formula A-8.

(...)

The maximum number of bytes per compressed image from the instantaneous bit-rate given in the previous table may be calculated using the following equation:

_Max_Compressed_Bytes = Max_Instantaneous_Bitrate / ( FrameRate × 8 )

(...)

Table A.47 – Example file size limitations for digital cinema applications including archiving

File size (in bytes)	2K digital cinema profile	4K digital cinema profile
Max compressed bytes for any image frame (aggregate of all 3 colour components)	Including bytes of possible PLT markers 16 fps: 1953125 24 fps: 1302083 25 fps: 1250000 30 fps: 1041666 48 fps : 651041 60 fps: 520833	Including bytes of possible PLT markers 16 fps: 1953125 24 fps: 1302083 25 fps: 1250000 30 fps: 1041666
Max compressed bytes for any single colour component of an image frame including all relevant tile-part headers.	Including bytes of possible PLT markers 16 fps: 1562500 24 fps: 1041666 25 fps: 1000000 30 fps: 833333 48 fps: 520833 60 fps: 416666	Including bytes of possible PLT markers (2K portion) 16 fps: 1562500 24 fps: 1041666 25 fps: 1000000 30 fps: 833333

[matmat]

There was an interesting thread on openjpeg list on how to use the library to extract the color component sizes: https://groups.google.com/forum/#!msg/openjpeg/_a8LvT4FrwA/xYuzfr7GAgAJ

This could probably also be done by using jpylyzer and looking at the psot element (and using the --format j2c option): https://jpylyzer.openpreservation.org/doc/latest/userManual.html#sot-marker

[remia]

Thanks for the analysis @matmat

So far we have been relying on asdcplib to check that the bitrate is under the specified limit, if I remember correctly, this only take the maximum size per frame (and not component) into account (then multiplying by the framerate to obtain the maximum bitrate).

We definitively need to start doing per frame analysis, do you think jpylyzer could be used to check for valid encoding of JP2K cinema profile frames ? Other validation tools use commercial JP2K decoder to do that (like Kakadu).

[bioleyl]

Have you made any progress on this point?

[remia]

Not that I'm aware of @bioleyl, are you having issue relating to these tests missing ?

As discussed above, it's probably not enough to rely only on MXF headers / asdcplib to perform this kind of tests, we would need to start parsing the JP2K stream which is more complex and involve introducing dependencies like jpylyzer. Also not clear whether encrypted essence would be able to be analysed or not.

This is certainly one of the main area that needs improvement at this time, related to #10.

[bioleyl]

Yes we got a DCP passing validation on Clairmeta and failing due to video asset bitrate on the new version of EasyDCP Player. If you want to start working on the feature, I can help testing on a whole bunch of DCP. We are validating for test purpose more than 10 DCP per day with your tool and we try to compare with other validating software like EasyDCP Player to see the differences.

[remia]

Thanks @bioleyl, would you have some DCPs showing this issue to share privately like trailers, so we have test content to develop ?

[bioleyl]

The only DCP with that problem that we have now is a 180GB DCP. I have cut with ffmpeg the part of the video asset which is problematic. It is now a mxf j2k of 130Mo. Is it good for your tests ? I don't have tools with me to produce or edit a DCP.

[remia]

That could a start yes, would be nice to have a look.

[jamiegau]

You could also do a quick creation of a DCP with DCP-O-Matic and specifically make it do higher then recommended bitrate for the video. That should result in a DCP that would trigger such a failure. Make sure you use NOISE for the input video so it is forced to max out to the peak bit rate selected.

If you want I could make one later today.

Or send me the 130MG slice of video. It's the original J2K encoded files I expect?

After creating a DCP around these files, I can run them through my online QC tool and send the PDF reports to those interested. However, the tool is due to update to the latest clairmeta release. A lot has been done since.

But yes, warning/error may need to be indicated. But this does bring up the question of if its a error or a warning as 500mbit streams are supported in all DCI-players these days. This also brings back the topic of working on a update to clairmeta so that specific application of all the tests are applied based on a set of target constraints. In that case, for example, you are in a position to flag error if the constraint specifies a profile that requires 250mbit or lower.

[jamiegau]

I have created a higher bitrate then 250 example. Easy to make with Resolve. Download from https://www.filemail.com/t/mJLwC8EH will work for 7 days. Below is the graph of my online tool of the file indicating its bitrate..

[remia]

Thanks for the DCP @jamiegau.

I noticed a typo in the checking of the issue date resulting in an error, will fix that asap. The IssueDate was not validated because there is no time zone in the date string and it was effectively issue after my current local time.

However this DCP is already detected as above maximum threshold by ClairMeta which was not the case @bioleyl reported where EasyDCP reported an error whereas ClairMeta didn't. Would be still interested to see such a sample.

[jamiegau]

Get the problematic example J2K stream to me and I'll make it into a DCP.

[bioleyl]

Here is the MXF https://www.swisstransfer.com/d/e7d8921f-8a8d-466a-9a4a-291c00bc198f

If you use Resolve with kakadu, I don't think you will be able to make a failing DCP. Resolve re-encode the video asset.

[remia]

Thanks @bioleyl got your file, will have a look at it.

[jamiegau]

Ok, got distracted.. This mxf file does not make cineasset happy, refuses to read it, and DCP-O-Matic insists it needs to be re-encoded.

Maybe I can use asdcplib to extract the j2k files directly. But I need to get get that all together.. Try tomorrow. Otherwise, this file is not helpful, however it was extracted, it was not done in a way easy to push back into a DCP.

[bioleyl]

I just cut the mxf asset with ffmpeg. If you have a better solution, I can do it differently.

[remia]

Sorry for the long overdue answer @bioleyl, I can not process your file either. It looks like it is an OP1a encoded MXF, which is what you would get from ffmpeg by default but not what is used for DCP IIRC, you could try using -f mxf_opatom before your output filename in the ffmpeg trim command.

[bioleyl]

I don't have access to that DCP anymore but next time I will try with mxf_opatom