ZJONSSON / node-unzipper

node.js cross-platform unzip using streams
Other
435 stars 116 forks source link

Unexpected end of file Zlib zlibOnError 5 Z_BUF_ERROR - caused by specific file #213

Open mottymilshtein opened 4 years ago

mottymilshtein commented 4 years ago

I am using this library to Parse a stream and extract from zip on the fly without storing the whole zip locally (stream to my server parse and send on entry to another server). I uploaded a zip with 250k images it fails on 'end of file Zlib zlibOnError 5 Z_BUF_ERROR' at the exact same entry every time, I removed this file and the parse works fine, I created a new zip with only that file and it fails every time (Using parse - for my code I must use parse since I don't store the whole file). I attached this file to the issue so you can check and see if there is a way to overcome this issue and perhaps skip it in flow, I tried to autodrain this specific entry but it mess up the next entry signature. p.s. I can open the file and view the image and also both in ubuntu and windows I can extract the file from the zip with no problem, the issue occurs only when I try to parse it from stream.

corrupted.zip

warerebel commented 3 years ago

There is something wrong with your zip file.

I use curl -o corrupted.zip https://github.com/ZJONSSON/node-unzipper/files/5149151/corrupted.zip

And this code

const fs = require("fs"); const unzip = require("unzipper"); let inStream = fs.createReadStream("./corrupted.zip");` inStream.on("open", ()=> { inStream.pipe(unzip.Parse()).on("entry", (entry) => console.log(entry.path)); });

I get result: invalid signature: 0x6d74683c

I recompressed your image file into mine.zip and ran the code again:

const fs = require("fs"); const unzip = require("unzipper"); let inStream = fs.createReadStream("./mine.zip");` inStream.on("open", ()=> { inStream.pipe(unzip.Parse()).on("entry", (entry) => console.log(entry.path)); });

I get result: b7c4d01c3200558d594134f3a320c5.jpeg mine.zip

mottymilshtein commented 3 years ago

We tried this image in multiple zip files (compressed on Ubuntu) and we get unexpected end of file especially if there are other images following this one, in debug I got invalid signature like you did, the main problem is that there is no error handling that enables me to skip this file and continue to the next one if the file inside the zip has invalid signature.

mottymilshtein commented 3 years ago

After checking it seems that with your zip I don't encounter the bug even if I add files to it but if I decompress it and compress it again in Ubuntu the combination of this specific file and the compression generates the end of file error, I still would like to have the ability to skip it because there is no issue with the other files -> I put it in a 250k images zip and I can't finish the extract but if I remove it from the zip I can, maybe it's a rare combination but still I can see that the end of file error appears in this library issues not once and would be nice to be able to skip files with bad signature

sliminality commented 3 years ago

I'm encountering this issue as well. It would be really nice to be able to skip over a failed file.

universe-42 commented 2 years ago

I am having same issue. After digging (zip within 5k files), i found that is exactly this image (see attach). Ubuntu unzip works fine.

Error: unexpected end of file
     at Zlib.zlibOnError [as onerror] (zlib.js:182:17)
     at Zlib.callbackTrampoline (internal/async_hooks.js:131:14)
 Error: Process exited with code 16
     at process.<anonymous> (/app/node_modules/@google-cloud/functions-framework/build/src/invoker.js:96:22)
     at process.emit (events.js:315:20)
     at process.EventEmitter.emit (domain.js:467:12)
     at process.exit (internal/process/per_thread.js:169:15)
     at sendCrashResponse (/app/node_modules/@google-cloud/functions-framework/build/src/logger.js:38:9)
     at process.<anonymous> (/app/node_modules/@google-cloud/functions-framework/build/src/invoker.js:92:44)
     at process.emit (events.js:315:20)
     at process.EventEmitter.emit (domain.js:467:12)
     at eventToObjectGenerator.promiseCreated (/app/node_modules/bluebird/js/release/debuggability.js:186:33)
     at activeFireEvent (/app/node_modules/bluebird/js/release/debuggability.js:229:44)
     at fireRejectionEvent (/app/node_modules/bluebird/js/release/debuggability.js:614:14)
     at Promise._notifyUnhandledRejection (/app/node_modules/bluebird/js/release/debuggability.js:61:9)
     at Async._drainQueue (/app/node_modules/bluebird/js/release/async.js:138:12)
     at Async._drainQueues (/app/node_modules/bluebird/js/release/async.js:146:10)
     at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/bluebird/js/release/async.js:17:14)
     at processImmediate (internal/timers.js:461:21)

Ubuntu 22.04 Node.js 14.15.4 It aslo failing in Google cloud Function (nodejs 14)

part-09-full.zip

hypertyper84 commented 1 year ago

This problem appears when the zip file contains another zip file.

aufabdulnafea commented 1 year ago

I am having the same error as well when compressing some a lot of files on Ubuntu, is there any solution for it?

sumedh-nimkarde-qatalog commented 1 year ago

Looks like this is happening when there is a zip inside zip

beala commented 3 months ago

Another archive that causes this. It does not contain a zip. https://drive.google.com/file/d/1LExvQ6kdwJITJqgzWIfgOax_3s72FrD9/view?usp=share_link

I switched to https://github.com/cthackers/adm-zip