Adding multiple large entries?

[gregory-h]

I'm hoping to use this to stream a tarball backup that would be too large to fit in memory.

My algorithm

• create the pack stream and pipe it to my http response

in an async repeat loop

• query a segment of data
• create a uniquely named pack.entry() and subscribe to its drain, finish and error events
• write a large json string to the entry, and on the entry.drain event end() the entry
• on the entry.finish event proceed to the next cycle of the async loop

at completion of the async loop

• pack.finalize()

The problem is that on my second itration of the async loop pack.entry(..) returns null, because this._destroyed is set at tar-stream/pack.js:110.

I see the issue pertaining to lack of support for multiple concurrent entries, which is why I've structured my algorithm to wait for the completion of one entry before starting another, but this does not seem to work - wondering if what I'm trying to do is achievable?

Thx,

[abrinckm]

@gregory-h try the solution here: https://github.com/mafintosh/tar-stream/issues/24

[bentorkington]

I have implemented this using async.eachOf. Here's a simplified example:

async.eachOf(files, (file, next) => {
    // pipe each file one at a time
    fs.createReadStream(file).pipe(pack.entry({ name: file }, (err) => next(err));
}, (err) => {
    // called once each iteration is done. todo: handle error here
    pack.finalize();
});