Closed joeshaw closed 9 years ago
Hmm, one possibility: section 4.3.14.1 of https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT says:
The value stored into the "size of zip64 end of central directory record" should be the size of the remaining record and should not include the leading 12 bytes.
By my count the record's total size is 56 bytes, so the "Size of record" field should be 44 (0x2c) rather than 56 (0x38).
Ok, opening a hex editor and changing that 0x38 to a 0x2c makes zipdetails happy, but it might be a red herring. zip -FF
fails in the same place regardless:
Local ( 1 7313354790): copying: 864 Pics 1.68gbs/20150114_202208.jpg (1875410 bytes)
Local ( 1 7315230282): copying: 864 Pics 1.68gbs/20150114_202212.jpg
zip I/O error: Bad address
zip error: Output file write failure (write error on zip file)
as does unzip -l
:
Archive: RFM-rgriffin.zip
error [RFM-rgriffin.zip]: missing 49368129 bytes in zipfile
(attempting to process anyway)
error [RFM-rgriffin.zip]: start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
zipdetails -v
output around the point where it dies:
1B4058E4A 000000004 50 4B 03 04 LOCAL HEADER #1029 04034B50
1B4058E4E 000000001 14 Extract Zip Spec 14 '2.0'
1B4058E4F 000000001 00 Extract OS 00 'MS-DOS'
1B4058E50 000000002 08 00 General Purpose Flag 0008
[Bit 3] 1 'Streamed'
1B4058E52 000000002 00 00 Compression Method 0000 'Stored'
1B4058E54 000000004 16 A5 4B 46 Last Mod Time 464BA516 'Wed Feb 11 20:40:44 2015'
1B4058E58 000000004 00 00 00 00 CRC 00000000
1B4058E5C 000000004 00 00 00 00 Compressed Length 00000000
1B4058E60 000000004 00 00 00 00 Uncompressed Length 00000000
1B4058E64 000000002 24 00 Filename Length 0024
1B4058E66 000000002 00 00 Extra Length 0000
1B4058E68 000000024 38 36 34 20 Filename '864 Pics
50 69 63 73 1.68gbs/20150114_202212.jpg'
20 31 2E 36
38 67 62 73
2F 32 30 31
35 30 31 31
34 5F 32 30
32 32 31 32
2E 6A 70 67
1B4058E8C 0001D3FFF ... PAYLOAD
1B422CE8B 000000004 50 4B 07 08 STREAMING DATA HEADER 08074B50
1B422CE8F 000000004 AF 15 10 F4 CRC F41015AF
1B422CE93 000000004 FF 3F 1D 00 Compressed Length 001D3FFF
1B422CE97 000000004 FF 3F 1D 00 Uncompressed Length 001D3FFF
I was able to decompress the whole zip without modification on a Linux box. I wonder if it's a Mac OS-only issue. unzip 6.00 on Linux versus 5.52 on OS X. I'll try installing a newer version on OS X and see if that's the issue.
Ok, what all of this has taught me is: large ZIP file support in OS X is garbage, and the failure modes are unkind.
The built-in archive utility doesn't support them, the version of unzip shipped with Yosemite doesn't include ZIP64 support, and the newer version that does support ZIP64 still blows up as soon as it hits 4 GB. Meanwhile, Linux works through the files happily.
The only thing that might make sense is to adjust the "size of zip64 end of central directory record" to make zipdetails
happy, but it doesn't seem to affect other tools (unzip, zipinfo) on Linux.
After your investigation I can't tell if there is anything for us to do here. Should I go ahead and close out this issue.
@ianlancetaylor: I think we should fix the size field in the ZIP64 end-of-central-directory record to match the spec. I'll write up a patch for that.
On 13 February 2015 at 09:09, Joe Shaw notifications@github.com wrote:
@ianlancetaylor https://github.com/ianlancetaylor: I think we should fix the size field in the ZIP64 end-of-central-directory record to match the spec. I'll write up a patch for that.
Sounds good. Please "git codereview mail https://golang.org/doc/contribute.html" it to adg@golang.org when it's ready.
I have an approximately 8 GB ZIP file created using archive/zip.
zipdetails -v
on the file ends with:The zip was built using Go 1.4 or 1.4.1 on a 64-bit Linux system.
A Go reader on OS X can reread the file just fine. This is my first foray into the internals of zip, so I am not sure how or why 56 is an unsupported size.
Smaller files (in the 3.7 GB range) are opened fine by
unzip
and the built-in OS X zip support.