Ymagis / ClairMeta

Clairmeta is a python package for Digital Cinema Package (DCP) probing and checking.
BSD 3-Clause "New" or "Revised" License
78 stars 21 forks source link

easydcp flags the lack of an <issuer> value in assetmap/pkl, but clairmeta does not. Why? #76

Closed Leanne93 closed 5 years ago

remia commented 5 years ago

Do you have any more details or example of AssetMap / PKL to show (and Clairmeta logs) ?

It should report the absence of Issuer tag when checking the XML file against XSD schema.

kieranjol commented 5 years ago

Hi we can share as much as we can regarding logs, but do you have an email that we could send the XML files to rather than sharing them here?

Leanne93 commented 5 years ago

This is a copy paste of the results of the script, I just changed some usernames/etc. for privacy's sake.

C:\Users\fakeuser>python -m clairmeta.cli check -type dcp J:\fakedrivename\fakedcpname\DCP
2019-02-25 15:00:33,977 - Clairmeta - ERROR - Could not intialize log file : [Error 3] The system cannot find the path specified: 'C:\\Users\\fakeuser/Library/Logs'
2019-02-25 15:00:34,072 - Clairmeta - INFO - Probing DCP : J:\fakedrivename\fakedcpname\DCP
2019-02-25 15:00:41,641 - Clairmeta - INFO - Total time : 7.57 seconds
2019-02-25 15:00:44,365 - Clairmeta - INFO - Checking DCP : J:\fakedrivename\fakedcpname\DCP
2019-02-25 15:03:00,625 - Clairmeta - INFO - DCP : J:\fakedrivename\fakedcpname\DCP
2019-02-25 15:03:00,628 - Clairmeta - INFO - Size : 11.71 GiB
2019-02-25 15:03:00,631 - Clairmeta - INFO - Warning(s) :
        check_dcnc_compliance - ContentTitle must have 12 parts, 11 found
2019-02-25 15:03:00,634 - Clairmeta - INFO - Total check : 53
2019-02-25 15:03:00,634 - Clairmeta - INFO - Total time : 136.26 sec
2019-02-25 15:03:00,637 - Clairmeta - INFO - Validation : Success

C:\Users\fakeuser>

C:\Users\fakeuser>python -m clairmeta.cli probe -type dcp J:\fakedrivename\fakedcpname\DCP
2019-02-25 15:04:47,520 - Clairmeta - ERROR - Could not intialize log file : [Error 3] The system cannot find the path specified: 'C:\\Users\\fakeuser/Library/Logs'
{'asset_list': {u'48339bb9-744e-420c-bad4-26dc60aa74d1': u'fakedcpname_Video.mxf',
                u'497eb4ef-dae4-452e-880d-ab784ebda1e3': u'CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
                u'554a0691-ff9d-41f8-95a7-62a218f12f49': u'PKL_554a0691-ff9d-41f8-95a7-62a218f12f49.xml',
                u'b323bee3-4472-42ec-abed-63332eb3800f': u'fakedcpname_Audio.mxf'},
 'assetmap_list': [{'FileName': 'ASSETMAP.xml',
                    'FilePath': 'J:\\fakedrivename\\fakedcpname\\DCP\\ASSETMAP.xml',
                    'Info': {'AssetMap': {u'AssetList': {u'Asset': [{u'ChunkList': {u'Chunk': {u'Length': 1248,
                                                                                               u'Offset': 0,
                                                                                               u'Path': u'PKL_554a0691-ff9d-41f8-95a7-62a218f12f49.xml',
                                                                                               u'VolumeIndex': 1}},
                                                                     u'Id': u'554a0691-ff9d-41f8-95a7-62a218f12f49',
                                                                     u'PackingList': True},
                                                                    {u'ChunkList': {u'Chunk': {u'Length': 1602,
                                                                                               u'Offset': 0,
                                                                                               u'Path': u'CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
                                                                                               u'VolumeIndex': 1}},
                                                                     u'Id': u'497eb4ef-dae4-452e-880d-ab784ebda1e3'},
                                                                    {u'ChunkList': {u'Chunk': {u'Length': 12320715338L,
                                                                                               u'Offset': 0,
                                                                                               u'Path': u'fakedcpname_Video.mxf',
                                                                                               u'VolumeIndex': 1}},
                                                                     u'Id': u'48339bb9-744e-420c-bad4-26dc60aa74d1'},
                                                                    {u'ChunkList': {u'Chunk': {u'Length': 253206126,
                                                                                               u'Offset': 0,
                                                                                               u'Path': u'fakedcpname_Audio.mxf',
                                                                                               u'VolumeIndex': 1}},
                                                                     u'Id': u'b323bee3-4472-42ec-abed-63332eb3800f'}]},
                                          u'Creator': u'OpenDCP 0.30.0',
                                          u'Id': u'4aaa82b0-6240-44c1-a63d-803873a756f2',
                                          u'IssueDate': u'2019-02-20T02:55:04+00:00',
                                          u'Issuer': '',
                                          'Schema': 'SMPTE',
                                          u'VolumeCount': 1}}}],
 'count_file': 4,
 'cpl_list': [{'FileName': 'CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
               'FilePath': 'J:\\fakedrivename\\fakedcpname\\DCP\\CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
               'Info': {'CompositionPlaylist': {u'AnnotationText': '',
                                                'AuxData': False,
                                                'ChannelConfiguration': 'Unknown',
                                                'ChannelCount': 2,
                                                'ChannelFormat': 0,
                                                u'ContentKind': u'short',
                                                u'ContentTitleText': u'fakedcpname_SHR-2D-24_F_EN-XX_IE-G_20-EN_2K_20190220_DCP_SMPTE_OV',
                                                u'ContentVersion': {u'Id': u'urn:uri:497eb4ef-dae4-452e-880d-ab784ebda1e3_2019-02-20T02:55:04+00:00',
                                                                    u'LabelText': u'497eb4ef-dae4-452e-880d-ab784ebda1e3_2019-02-20T02:55:04+00:00'},
                                                u'Creator': u'OpenDCP 0.30.0',
                                                'D-BOX': False,
                                                'DecompositionLevels': 5,
                                                'DolbyVision': False,
                                                'EclairColor': False,
                                                'EditRate': 24,
                                                'Encrypted': False,
                                                'FrameRate': 24,
                                                'HighFrameRate': False,
                                                u'Id': u'497eb4ef-dae4-452e-880d-ab784ebda1e3',
                                                u'IssueDate': u'2019-02-20T02:55:04+00:00',
                                                u'Issuer': '',
                                                'Markers': False,
                                                'PKLId': u'554a0691-ff9d-41f8-95a7-62a218f12f49',
                                                'Picture': True,
                                                'Precincts': 6,
                                                u'RatingList': '',
                                                u'ReelList': [{'AnnotationText': '',
                                                               'Assets': {'Picture': {u'AnnotationText': u'fakedcpname_Video.mxf',
                                                                                      'CPLEntryPoint': 0,
                                                                                      'CPLOutPoint': 21064,
                                                                                      u'Duration': 21064,
                                                                                      u'EditRate': 24,
                                                                                      'Encrypted': False,
                                                                                      u'EntryPoint': 0,
                                                                                      u'FrameRate': 24,
                                                                                      'HighFrameRate': False,
                                                                                      u'Id': u'48339bb9-744e-420c-bad4-26dc60aa74d1',
                                                                                      u'IntrinsicDuration': 21064,
                                                                                      'OutPoint': 21064,
                                                                                      'Path': u'fakedcpname_Video.mxf',
                                                                                      'Probe': {u'1': u'128x128',
                                                                                                u'2': u'256x256',
                                                                                                u'3': u'256x256',
                                                                                                u'4': u'256x256',
                                                                                                u'5': u'256x256',
                                                                                                u'6': u'256x256',
                                                                                                u'AspectRatio': 1.85,
                                                                                                u'AssetUUID': u'48339bb9-744e-420c-bad4-26dc60aa74d1',
                                                                                                u'AverageBitRate': 112.3,
                                                                                                u'CodeblockHeight': 3,
                                                                                                u'CodeblockStyle': 0,
                                                                                                u'CodeblockWidth': 3,
                                                                                                u'CompanyName': u'OpenDCP',
                                                                                                u'ContainerDuration': 21064,
                                                                                                u'DecompositionLevels': 5,
                                                                                                u'EditRate': 24,
                                                                                                u'EncryptedEssence': False,
                                                                                                u'LabelSetType': u'SMPTE',
                                                                                                u'MaxBitRate': 125,
                                                                                                u'MultiCompTransform': 1,
                                                                                                u'NumberOfLayers': 1,
                                                                                                u'Precincts': 6,
                                                                                                u'ProductName': u'OpenDCP',
                                                                                                u'ProductUUID': u'43059a1d-0432-4101-b83f-736815acf31d',
                                                                                                u'ProductVersion': u'0.30.0',
                                                                                                u'ProgressionOrder': 4,
                                                                                                'Resolution': '1998x1080',
                                                                                                u'Rsize': 3,
                                                                                                u'SPqcd': u'972096f096f096c08f008f008ee087508750876870057005704777d377d37762',
                                                                                                u'SampleRate': 24,
                                                                                                u'Scod': 1,
                                                                                                u'Sqcd': 66,
                                                                                                u'StoredHeight': 1080,
                                                                                                u'StoredWidth': 1998,
                                                                                                u'Transformation': 0,
                                                                                                u'XOsize': 0,
                                                                                                u'XTOsize': 0,
                                                                                                u'XTsize': 1998,
                                                                                                u'Xsize': 1998,
                                                                                                u'YOsize': 0,
                                                                                                u'YTOsize': 0,
                                                                                                u'YTsize': 1080,
                                                                                                u'Ysize': 1080},
                                                                                      u'ScreenAspectRatio': 1.85,
                                                                                      'Stereoscopic': False,
                                                                                      'TimeCodeDuration': '00:14:37:16',
                                                                                      'TimeCodeIn': '00:00:00:00',
                                                                                      'TimeCodeOut': '00:14:37:16'},
                                                                          'Sound': {u'AnnotationText': u'fakedcpname_Audio.mxf',
                                                                                    'CPLEntryPoint': 0,
                                                                                    'CPLOutPoint': 21064,
                                                                                    u'Duration': 21064,
                                                                                    u'EditRate': 24,
                                                                                    'Encrypted': False,
                                                                                    u'EntryPoint': 0,
                                                                                    u'Id': u'b323bee3-4472-42ec-abed-63332eb3800f',
                                                                                    u'IntrinsicDuration': 21064,
                                                                                    'OutPoint': 21064,
                                                                                    'Path': u'fakedcpname_Audio.mxf',
                                                                                    'Probe': {u'AssetUUID': u'b323bee3-4472-42ec-abed-63332eb3800f',
                                                                                              'AudioAnalyze': {'pk_lvl_db': u'-1.40|-1.71',
                                                                                                               'pk_lvl_db_overall': u'-1.40',
                                                                                                               'rms_lvl_db': u'-27.73|-27.68',
                                                                                                               'rms_lvl_db_overall': u'-27.71'},
                                                                                              u'AudioSamplingRate': 48000,
                                                                                              u'AvgBps': 288000,
                                                                                              u'BlockAlign': 6,
                                                                                              u'ChannelCount': 2,
                                                                                              u'ChannelFormat': 0,
                                                                                              u'CompanyName': u'OpenDCP',
                                                                                              u'ContainerDuration': 21064,
                                                                                              u'EditRate': 24,
                                                                                              u'EncryptedEssence': False,
                                                                                              u'LabelSetType': u'SMPTE',
                                                                                              u'LinkedTrackID': 2,
                                                                                              u'Locked': 0,
                                                                                              u'ProductName': u'OpenDCP',
                                                                                              u'ProductUUID': u'43059a1d-0432-4101-b83f-736815acf31d',
                                                                                              u'ProductVersion': u'0.30.0',
                                                                                              u'QuantizationBits': 24},
                                                                                    'TimeCodeDuration': '00:14:37:16',
                                                                                    'TimeCodeIn': '00:00:00:00',
                                                                                    'TimeCodeOut': '00:14:37:16'}},
                                                               'Id': u'fff722bb-5149-4617-91b7-3a0d62035e08',
                                                               'Position': 1}],
                                                'Resolution': '1998x1080',
                                                'Schema': 'SMPTE',
                                                'ScreenAspectRatio': 1.85,
                                                'Sound': True,
                                                'SoundLanguage': 'Unknown',
                                                'Stereoscopic': False,
                                                'Subtitle': False,
                                                'SubtitleLanguage': 'Unknown',
                                                'TotalDuration': 21064,
                                                'TotalTimeCodeDuration': '00:14:37:16'}}}],
 'package_type': 'OV',
 'path': 'J:\\fakedrivename\\fakedcpname\\DCP',
 'pkl_list': [{'FileName': 'PKL_554a0691-ff9d-41f8-95a7-62a218f12f49.xml',
               'FilePath': 'J:\\fakedrivename\\fakedcpname\\DCP\\PKL_554a0691-ff9d-41f8-95a7-62a218f12f49.xml',
               'Info': {'PackingList': {u'AnnotationText': '',
                                        u'AssetList': {u'Asset': [{u'AnnotationText': u'fakedcpname_Video.mxf',
                                                                   u'Hash': u'AGipzkzd0b477F6GW7rU3KbnygU=',
                                                                   u'Id': u'48339bb9-744e-420c-bad4-26dc60aa74d1',
                                                                   u'OriginalFileName': u'fakedcpname_Video.mxf',
                                                                   'Path': u'fakedcpname_Video.mxf',
                                                                   u'Size': 12320715338L,
                                                                   u'Type': u'application/mxf'},
                                                                  {u'AnnotationText': u'fakedcpname_Audio.mxf',
                                                                   u'Hash': u'A2RWKoa4Z3bmhuAG6nGdxM1AxcE=',
                                                                   u'Id': u'b323bee3-4472-42ec-abed-63332eb3800f',
                                                                   u'OriginalFileName': u'fakedcpname_Audio.mxf',
                                                                   'Path': u'fakedcpname_Audio.mxf',
                                                                   u'Size': 253206126,
                                                                   u'Type': u'application/mxf'},
                                                                  {u'Hash': u'QRIJW0OK86YIkPvZKRb2dfk8Smc=',
                                                                   u'Id': u'497eb4ef-dae4-452e-880d-ab784ebda1e3',
                                                                   u'OriginalFileName': u'CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
                                                                   'Path': u'CPL_497eb4ef-dae4-452e-880d-ab784ebda1e3.xml',
                                                                   u'Size': 1602,
                                                                   u'Type': u'text/xml'}]},
                                        u'Creator': u'OpenDCP 0.30.0',
                                        u'Id': u'554a0691-ff9d-41f8-95a7-62a218f12f49',
                                        u'IssueDate': u'2019-02-20T02:55:04+00:00',
                                        u'Issuer': '',
                                        'Schema': 'SMPTE'}}}],
 'schema': 'SMPTE',
 'size': '11.71 GiB',
 'type': 'DCP',
 'volindex_list': [{'FileName': 'VOLINDEX.xml',
                    'FilePath': 'J:\\fakedrivename\\fakedcpname\\DCP\\VOLINDEX.xml',
                    'Info': {'VolumeIndex': {u'Index': 1,
                                             'Schema': 'SMPTE'}}}]}

C:\Users\fakeuser>
remia commented 5 years ago

To sumup, I don't think there is any problem with an empty Issuer tag in the PKL / AM (for the CPL the XML tag is optional). I didn't get feedback on any issue related to this but additional experience is always welcome.

I think we could implement a check (as a warning) for empty descriptive fields like this (Issuer, Creator, ..).

kieranjol commented 5 years ago

Cool, my main worry would have been the DCP getting rejected by a very standards compliant server. But perhaps an empty element is more compliant that a completely missing element..

remia commented 5 years ago

Yes, when strictly applying the XML schema from the standard, it's valid because only the tag is required not the content.

It's unlikely that any server would reject a DCP based on this because it's valid per the SMPTE standard. But I can't guarantee this of course (easyDCP treating this only as a warning is another hint).

remia commented 5 years ago

Just added additionnals checks for empty Issuer / Creator / AnnotationText fields in the AssetMap, PKL and CPL. These are warning by default.

See #89

kieranjol commented 5 years ago

Thank you!