Cannot extract zipfile contents using default Windows File Explorer

[mecampbellsoup]

OK @evanmiller - this is a weird one! Hoping to hear your thoughts on the issue, but I don't expect it to be something related to mod_zip per se; but I am hopeful it piques your interest... :

Summary

Zipfiles generated by mod_zip can be extracted using non-default Windows extraction software like 7-Zip, but cannot be extracted using the default Windows OS File Explorer application.

Description

We use mod_zip to serve zipfiles directly from NGINX - we love it! Really happy to have more static file serving logic moved off our application servers and onto the web server where it belongs.

However, after switching to the mod_zip-based implementation for zipfile serving, we started noticing something very strange.

For Windows-based clients (I have reproduced the issue on Windows 10 specifically), when attempting to extract the zipfile contents using the built-in/default File Explorer application, an error is encountered along the lines of: "The Compressed (zipped) Folder '[directory\\\file name.zip]' is invalid.

However, when using a different zip application (7-Zip), I am able to extract the file contents without any issue whatsoever:

Like I said, I doubt this is related to mod_zip, but I figured I'd ask - maybe there is some file header present that isn't compatible with the zipfile implementation that File Explorer uses, which we may be able to configure somehow...?

[mecampbellsoup]

When testing the file w/ 7-Zip's test utility, the file is reported to be valid.

[mecampbellsoup]

(⎈ kubernetes-admin@chi.local:concierge-prod)➜  k8s-billing git:(master) ✗ file -z /Users/mattysoup/Downloads/352506-cube-001.zip
/Users/mattysoup/Downloads/352506-cube-001.zip: ERROR:[gzip: Unknown compression format] (Zip archive data, at least v1.0 to extract)

Possible cause (unknown compression format)?

[mecampbellsoup]

@evanmiller I believe I've gotten to the root of the issue - the zipfile is basically uncompressed, despite being, well, a "compressed" zipfile. (We looked at the compression method bytes in a problematic file and its value was set to 0 i.e. no compression.)

I suppose it makes sense that Windows default archive extractor wasn't written to handle not-actually-compressed archive files...

The reason our zipfiles are not compressed is because they contain image files which are already in efficiently compressed formats themselves.

Will revert as I continue digging into this.

[dup2]

It seems like #23 says that the ZIP files are not compressed by design.

@evanmiller Is this true? Could this be added to the Readme?

[mecampbellsoup]

@dup2 thank you.

I ended up discovering the source of my issue - naturally it was Operator Error 😆

The ngzip library I am using removes common prefixes for files contained in the manifest file, so as to remove superfluous leading directory paths inside the uncompressed file. This is a Good Thing since those otherwise empty leading directory paths serve no function in the expanded zipfile.

However, my manifest was being written like the following:

- 0 %2Fmnt%2Fdata%2Fuserdata%2Ftest%2F311ccef4b01d05d3e02415dd7cf6d46d02af694790ec37c7b1caa33750065c9b%2Foutputs%2Fblender%2F2.82%2F#{job.id}-K%C3%BAca%2F%2F0001.png /0001.png

Note the leading forward slash preceding the in-expanded-directory file path /0001.png.

As Windows uses \ instead of / for file path delimiters, including / in a file name results in an invalid Windows file path. It seems that the default File Explorer encounters such an error (i.e. try renaming a file in Windows to include / and observe the resultant error) and fails to extract the zipfile contents without propagating any sort of descriptive error. Other applications like 7-Zip seem to have encountered this and handle them more gracefully, i.e. simply removing the leading forward slash to make the extracted file name valid.

Once I adjusted my manifest output to the following I was able to get Windows extractions working:

- 0 %2Fmnt%2Fdata%2Fuserdata%2Ftest%2F311ccef4b01d05d3e02415dd7cf6d46d02af694790ec37c7b1caa33750065c9b%2Foutputs%2Fblender%2F2.82%2F#{job.id}-K%C3%BAca%2F0001.png 0001.png