Stuk / jszip

Create, read and edit .zip files with Javascript
https://stuk.github.io/jszip/
Other
9.77k stars 1.3k forks source link

Large zip file breaks with loadAsync #912

Open velgajski1 opened 11 months ago

velgajski1 commented 11 months ago

Tested on Chrome/Edge/Safari.

Error is : DOMException: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.

Same file works with zip.js for example, and it used to work with jszip until few days ago. File size is around 3GB. Tested on smaller (100mb) zip files and for those it still works normally.

Any clues?

sobjornstad commented 2 weeks ago

I tried to isolate this by creating a test with the same file in a Node script and got instead the following error:

/.../node_modules/jszip/lib/reader/DataReader.js:26
            throw new Error("End of data reached (data length = " + this.length + ", asked index = " + (newIndex) + "). Corrupted zip ?");
                  ^

Error: End of data reached (data length = 4026263362, asked index = -270818191). Corrupted zip ?
    at NodeBufferReader.checkIndex (/.../node_modules/jszip/lib/reader/DataReader.js:26:19)
    at NodeBufferReader.setIndex (/.../node_modules/jszip/lib/reader/DataReader.js:35:14)
    at ZipEntries.readCentralDir (/.../node_modules/jszip/lib/zipEntries.js:132:21)
    at ZipEntries.load (/.../node_modules/jszip/lib/zipEntries.js:256:14)
    at /.../node_modules/jszip/lib/load.js:48:24

unzip -t says the file is valid, though, which makes me wonder if it's a different manifestation of this bug that for some reason occurs differently when reading from a buffer in Node: https://github.com/Stuk/jszip/issues/543

(Could always be two different issues though.)

Test script (zero points for elegance, it's multiple levels of Claude-generated code and editing while debugging and is more complicated than it needs to be...):

import JSZip from 'jszip';

import * as fs from 'fs';
import * as path from 'path';

class NodeFile {
  constructor(filePath) {
    const stats = fs.statSync(filePath);
    this.name = path.basename(filePath);
    this.size = stats.size;
    this.type = ''; // MIME type, you may want to implement this
    this.lastModified = stats.mtime.getTime();
    this._path = filePath;
  }

  stream() {
    return fs.createReadStream(this._path);
  }

  slice(start, end, contentType) {
    // This is a simplified implementation
    const stream = fs.createReadStream(this._path, { start, end });
    return new NodeFile(this._path, stream, contentType);
  }
}

function stream2buffer(stream) {
    return new Promise((resolve, reject) => {
        const _buf = [];
        stream.on("data", (chunk) => _buf.push(chunk));
        stream.on("end", () => resolve(Buffer.concat(_buf)));
        stream.on("error", (err) => reject(err));
    });
}

const myFile = new NodeFile("my_file_path");
const buff = stream2buffer(myFile.stream());
JSZip.loadAsync(buff);