101arrowz / fflate

High performance (de)compression in an 8kB package
https://101arrowz.github.io/fflate
MIT License
2.27k stars 79 forks source link

unzip does not support >65,535 files #137

Closed neilrackett closed 2 years ago

neilrackett commented 2 years ago

How to reproduce

If you try to unzip or unzipSync a zip file containing >65,535 files, only 65,535 files will be listed.

For example:

// Load zip file containing 100,000 files into ArrayBuffer
const zipArray = new Uint8Array(zipBuffer);
const files = unzipSync(zipArray);

console.log(Object.keys(files).length) // 65535

The problem

This issue occurs without any error messages, the number of files is simply truncated to 65,535.

101arrowz commented 2 years ago

Could you provide a sample file (or how to generate one) so I can reproduce this?

neilrackett commented 2 years ago

No problem: here's a zip file containing 70,000 sequentially named text files I created using a simple node script, then zipped using tar.

const fs = require('fs');
for (let i = 1; i <= 70000; i++) {
  fs.writeFileSync(`./files/file-${('000000' + i).substr(-6)}.txt`, 'x');
}
tar -a -c -f ../70000-files.zip *

70000-files.zip

101arrowz commented 2 years ago

fflate isn't looking at the Zip64 end-of-central-directory record so it can only use the number of files given in the standard 16-bit EOCD record. This should be fixable, I'll look into it.

101arrowz commented 2 years ago

Fixed in v0.7.4. Sorry for the long delay!

neilrackett commented 2 years ago

Amazing, thank you!