I have attempted to trigger the error message for PKG-8 which states that encrypted files should be stored and not compressed. I have not been able to trigger this error message, and my attempts have brought up a few issues,
This issue is raised in relation to PKG-8, but of course we also have POL_1 which should fail on ALL encrypted files. This is not always triggered either, so I am raising that as a separate issue.
As far as I can tell, the trigger for PKG-8 is that a file in the zip archive should be registered in the manifest, be encrypted and not be stored. I tried a couple of scenarios to trigger this, and they produced a variety of results, but no PKG-8.
Scenario 1: generating APP-2 error
I encrypted the styles.xml file which already existed in the zip archive and is registered in the manifest.
The validator doesn't actually validate the file though based on the log:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001a.ods"
APP-1: [INFO] Validating filer\testfiler\E001a.ods.
APP-2: [ERROR] Unsupported feature encryption used in entry styles.xml
I can see a couple of issues with this error message:
It is not PKG-8 or POL_1.
APP-2 should be “Supplied path %s IS NOT an existing file", so it is unclear how we are getting this result here.
Since this is an APP-x message, what will happen when this is validated using API?
Above is the result for running the validator with profile. Without profile, the log looks like this:
C:\odf\odf-validator-main>odf-validator.bat "filer\testfiler\E001a.ods"
APP-1: [INFO] Validating filer\testfiler\E001a.ods.
org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: Unsupported feature encryption used in entry styles.xml
at org.apache.commons.compress.archivers.zip.ZipUtil.checkRequestedFeatures(ZipUtil.java:147)
at org.apache.commons.compress.archivers.zip.ZipFile.getInputStream(ZipFile.java:953)
at org.openpreservation.format.zip.ZipFileProcessor.getEntryInputStream(ZipFileProcessor.java:116)
at org.openpreservation.odf.pkg.PackageParserImpl.processEntry(PackageParserImpl.java:129)
at org.openpreservation.odf.pkg.PackageParserImpl.processZipEntries(PackageParserImpl.java:109)
at org.openpreservation.odf.pkg.PackageParserImpl.parsePackage(PackageParserImpl.java:100)
at org.openpreservation.odf.pkg.PackageParserImpl.parsePackage(PackageParserImpl.java:70)
at org.openpreservation.odf.validation.ValidatingParserImpl.parsePackage(ValidatingParserImpl.java:74)
at org.openpreservation.odf.validation.Validator.validatePackage(Validator.java:107)
at org.openpreservation.odf.validation.Validator.validate(Validator.java:83)
at org.openpreservation.odf.apps.CliValidator.validatePath(CliValidator.java:68)
at org.openpreservation.odf.apps.CliValidator.call(CliValidator.java:60)
at org.openpreservation.odf.apps.CliValidator.call(CliValidator.java:35)
at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at org.openpreservation.odf.apps.CliValidator.main(CliValidator.java:87)
Scenario 2: not generating error message related to encryption
Here, I encrypted the thumbnail instead. Again, this was already in the ODS-package and registered in the manifest.
E001e.ods
Again, this should have triggered PKG-8 since the file is in the manifest:
And the attributes of the zip entry include encryption and not stored:
Now the validator is happy to start validating the file and produces this log:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001e.ods"
APP-1: [INFO] Validating filer\testfiler\E001e.ods.
APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001e.ods.
MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file.
DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected
POL_2: E001e.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3".
NOT VALID, 2 errors, 0 warnings and 1 info messages.
Again, some issues with this result:
Not PKG-8 or POL_1
Although there are errors, they are not related to encryption at all, i.e. the file is allowed through. The error is just because of the mimetype file position, and this can be corrected.
Why is there a discrepancy in how this is treated compared to scenario 1? They are two different files, but the encryption and compression is the same.
In this case, the error messages (except of course for POL_2) are the same when validator is run without the profile.
Scenario 3: Adding a new random file to the zip archive
So in the first two scenarios, I used files that were already present in the zip archive. For this one, I tried adding the random extra file random.png (which is encrypted) to the root of the zip archive. I didn't add it to the manifest.
This generated the following:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001f.ods"
APP-1: [INFO] Validating filer\testfiler\E001f.ods.
APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001f.ods.
MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file.
DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected
MAN-1: random.png [ERROR] The manifest SHALL contain an entry for every file in the package, zip entry random.png has no corresponding manifest file entry.
POL_2: E001f.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3".
NOT VALID, 3 errors, 0 warnings and 1 info messages.
This will produce an error message since the file is not in the manifest, but no errors related to encryption.
Scenario 4: Adding random.png to the manifest
So I added random.png to the manifest:
and tried again:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001g.ods"
APP-1: [INFO] Validating filer\testfiler\E001g.ods.
APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001g.ods.
MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file.
DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected
POL_2: E001g.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3".
NOT VALID, 2 errors, 0 warnings and 1 info messages.
And now it gets through (again except for the mimetype error).
Scenario 5: Trying with a file names signatures in META-INF/
Since there are some special restrictions on the META-INF/ folder, I tried adding an encrypted file named signatures to this folder. In this folder, it doesn't need to be in the manifest, so I didn't add it there.
The result was like scenario 1:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001h.ods"
APP-1: [INFO] Validating filer\testfiler\E001h.ods.
APP-2: [ERROR] Unsupported feature encryption used in entry META-INF/signatures
And without the profile, it also reacted the same as in scenario 1.
The signatures file is actually the same file as random.png - I just renamed it.
Summary
I have not been able to generate PKG-8, but my attempts showed that there is an APP-2 error message in some cases, and the file slips through in other cases.
I have attempted to trigger the error message for PKG-8 which states that encrypted files should be stored and not compressed. I have not been able to trigger this error message, and my attempts have brought up a few issues,
This issue is raised in relation to PKG-8, but of course we also have POL_1 which should fail on ALL encrypted files. This is not always triggered either, so I am raising that as a separate issue.
As far as I can tell, the trigger for PKG-8 is that a file in the zip archive should be registered in the manifest, be encrypted and not be stored. I tried a couple of scenarios to trigger this, and they produced a variety of results, but no PKG-8.
Scenario 1: generating APP-2 error I encrypted the styles.xml file which already existed in the zip archive and is registered in the manifest.
E001a.ods
This should have triggered PKG-8 since the file is in the manifest as:
And the attributes of the zip entry include encryption and not stored:
Path = styles.xml Folder = - Size = 20310 Packed Size = 3190 Modified = 2024-04-30 08:01:14.0000000 Created = Accessed = Attributes = Encrypted = + Comment = CRC = 7BBE0528 Method = pkAES-256 Deflate Characteristics = NTFS StrongCrypto : Encrypt StrongCrypto UTF8 Host OS = FAT Version = 51 Volume Index = 0 Offset = 19770
The validator doesn't actually validate the file though based on the log:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001a.ods" APP-1: [INFO] Validating filer\testfiler\E001a.ods. APP-2: [ERROR] Unsupported feature encryption used in entry styles.xml
I can see a couple of issues with this error message:
C:\odf\odf-validator-main>odf-validator.bat "filer\testfiler\E001a.ods" APP-1: [INFO] Validating filer\testfiler\E001a.ods. org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: Unsupported feature encryption used in entry styles.xml at org.apache.commons.compress.archivers.zip.ZipUtil.checkRequestedFeatures(ZipUtil.java:147) at org.apache.commons.compress.archivers.zip.ZipFile.getInputStream(ZipFile.java:953) at org.openpreservation.format.zip.ZipFileProcessor.getEntryInputStream(ZipFileProcessor.java:116) at org.openpreservation.odf.pkg.PackageParserImpl.processEntry(PackageParserImpl.java:129) at org.openpreservation.odf.pkg.PackageParserImpl.processZipEntries(PackageParserImpl.java:109) at org.openpreservation.odf.pkg.PackageParserImpl.parsePackage(PackageParserImpl.java:100) at org.openpreservation.odf.pkg.PackageParserImpl.parsePackage(PackageParserImpl.java:70) at org.openpreservation.odf.validation.ValidatingParserImpl.parsePackage(ValidatingParserImpl.java:74) at org.openpreservation.odf.validation.Validator.validatePackage(Validator.java:107) at org.openpreservation.odf.validation.Validator.validate(Validator.java:83) at org.openpreservation.odf.apps.CliValidator.validatePath(CliValidator.java:68) at org.openpreservation.odf.apps.CliValidator.call(CliValidator.java:60) at org.openpreservation.odf.apps.CliValidator.call(CliValidator.java:35) at picocli.CommandLine.executeUserObject(CommandLine.java:2041) at picocli.CommandLine.access$1500(CommandLine.java:148) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) at picocli.CommandLine.execute(CommandLine.java:2170) at org.openpreservation.odf.apps.CliValidator.main(CliValidator.java:87)
Scenario 2: not generating error message related to encryption Here, I encrypted the thumbnail instead. Again, this was already in the ODS-package and registered in the manifest. E001e.ods
Again, this should have triggered PKG-8 since the file is in the manifest:
And the attributes of the zip entry include encryption and not stored:
Path = Thumbnails\thumbnail.png Folder = - Size = 15196 Packed Size = 15110 Modified = 2024-04-30 08:01:14.0000000 Created = Accessed = Attributes = A Encrypted = + Comment = CRC = 8B1062C5 Method = pkAES-256 Deflate Characteristics = NTFS StrongCrypto : Encrypt StrongCrypto Host OS = FAT Version = 51 Volume Index = 0 Offset = 747
Now the validator is happy to start validating the file and produces this log:
C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001e.ods" APP-1: [INFO] Validating filer\testfiler\E001e.ods. APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001e.ods. MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file. DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected POL_2: E001e.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3". NOT VALID, 2 errors, 0 warnings and 1 info messages.
Again, some issues with this result:
Scenario 3: Adding a new random file to the zip archive So in the first two scenarios, I used files that were already present in the zip archive. For this one, I tried adding the random extra file random.png (which is encrypted) to the root of the zip archive. I didn't add it to the manifest.
E001f.ods
This generated the following: C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001f.ods" APP-1: [INFO] Validating filer\testfiler\E001f.ods. APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001f.ods. MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file. DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected MAN-1: random.png [ERROR] The manifest SHALL contain an entry for every file in the package, zip entry random.png has no corresponding manifest file entry. POL_2: E001f.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3". NOT VALID, 3 errors, 0 warnings and 1 info messages.
This will produce an error message since the file is not in the manifest, but no errors related to encryption.
Scenario 4: Adding random.png to the manifest So I added random.png to the manifest:
E001g.ods
and tried again: C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001g.ods" APP-1: [INFO] Validating filer\testfiler\E001g.ods. APP-5: [INFO] DNA ODF Spreadsheets Preservation Specification Profile report for filer\testfiler\E001g.ods. MIM-1: mimetype [ERROR] The "mimetype" file SHALL be the first file of the zip file. DOC-3: mimetype [INFO] OpenDocument MIMETYPE application/vnd.oasis.opendocument.spreadsheet detected POL_2: E001g.ods [ERROR] Standard Compliance | The file MUST comply with the standard "OASIS Open Document Format for Office Applications (OpenDocument) v1.3". NOT VALID, 2 errors, 0 warnings and 1 info messages.
And now it gets through (again except for the mimetype error).
Scenario 5: Trying with a file names signatures in META-INF/ Since there are some special restrictions on the META-INF/ folder, I tried adding an encrypted file named signatures to this folder. In this folder, it doesn't need to be in the manifest, so I didn't add it there.
E001h.ods
The result was like scenario 1: C:\odf\odf-validator-main>odf-validator.bat -p "filer\testfiler\E001h.ods" APP-1: [INFO] Validating filer\testfiler\E001h.ods. APP-2: [ERROR] Unsupported feature encryption used in entry META-INF/signatures
And without the profile, it also reacted the same as in scenario 1.
The signatures file is actually the same file as random.png - I just renamed it.
Summary I have not been able to generate PKG-8, but my attempts showed that there is an APP-2 error message in some cases, and the file slips through in other cases.