DanielHindi / aws-s3-zipper

takes an amazon s3 bucket folder and zips it for streaming or serializes to a file
119 stars 74 forks source link

Piping streamZipDataTo directly to Express response results in corrupt file #19

Open ibarsi opened 6 years ago

ibarsi commented 6 years ago

I've got the following code:

app.all('/', function (req, res) {
    const options = {
        pipe: res,
        folderName: `/my/S3/folder`,
        recursive: true,
    };

    res.set('Content-Type', 'application/zip, application/octet-stream');
    res.setHeader('Content-disposition', 'attachment; filename=myZip.zip');

    return q.ninvoke(zipper, 'streamZipDataTo', options)
});

The response is received and interpreted as a zip file successfully. However, the file is unable to be opened as if it was corrupted.

I've tested the same code using zipToFile and the zip file saved locally can be opened and contains the right data. Furthermore, the streamed file is significantly smaller than the locally stored alternative. Its almost as if the response is ended too early?

The zip file size I've been testing with is just under 5mb. Not sure if the stream is chunked, but not cycled to the end? I'm just speculating here, I haven't had the chance to clone and test locally but I will if I come around to it in the near future.

In the mean time, do you have any ideas as to what the issue could be?

kausikMR commented 4 months ago

I too have the same issue with, any help will be appreciatable

s3Zipper.streamZipDataTo(
      {
        pipe: res,
        folderName: folderName,
        recursive: true,
      },
      function (err: any, result: any) {
        console.log('result ', result)
        if (err) {
          console.error(err)
        } else {
          console.log('result ', result)
        }
      }
    )
kausikMR commented 4 months ago

the issue is I forgot to change the response type as stream in frontend, fixing that resolves this issue. 🎉

const response = http.get('/download-url', {responseType: 'stream'});