archiverjs / node-archiver

a streaming interface for archive generation
https://www.archiverjs.com
MIT License
2.76k stars 218 forks source link

Zip.append callback is not invoked. #385

Open tom-marsh opened 4 years ago

tom-marsh commented 4 years ago

The documentation for the Zip.append method states that it takes a callback function, however the callback function is never actually called.

Minimal example:

"use strict";

const archiver = require("archiver");
const Writable = require("stream").Writable;

const archive = archiver("zip", {
    zlib: {
        level: 9
    }
});

archive.pipe(new Writable({
    write(chunk, encoding, callback) {
        console.log("Processing:", chunk);
        callback();
    }
}));

archive.append("abc", {
    name: "abc.txt"
}, () => {
    // Never logged.
    console.log("Done.");
});

archive.finalize();

Looking at the source code, the append method in lib/plugins/zip.js has a callback function, which seems to be provided by _moduleAppend in lib/core.js, rather than by the user.

The desired use case was wrapping the archiver in a Node writable stream as the final step in a stream processing pipeline for exporting data, for which callbacks are a cleaner interface to manage backpressure than events.

tracycollins commented 4 years ago

Yes, just stumbled on this, too. Would really help implementing backpressure.

roberttolton commented 4 years ago

Am having this issue too, within a Google Cloud Function:

archive.append(remoteFile.createReadStream()
    .on('error', (error) => {
        console.error(error);
        reject(error);
        throw new functions.https.HttpsError('unknown', `Error streaming ${filename}`);
    })
    .on('response', (response) => {
        // Server connected and responded with the specified status and headers.
    })
    .on('end', () => {
        console.log(`File ${filename} completed streaming.`);
        resolve(filename);
    }), {
    name: filename
}, () => {
    console.log(`File ${filename} appended to ZIP.`);
    resolve(filename);
});

The callback is never reached, so for ages I was wondering why my promise (not in the code above) was never resolved. So right now I'm resolving the promise when the data is downloaded, not when I know the append was successful. It works, but it's not how it should be..