Open mottymilshtein opened 4 years ago
There is something wrong with your zip file.
I use curl -o corrupted.zip https://github.com/ZJONSSON/node-unzipper/files/5149151/corrupted.zip
And this code
const fs = require("fs");
const unzip = require("unzipper");
let inStream = fs.createReadStream("./corrupted.zip");`
inStream.on("open", ()=> {
inStream.pipe(unzip.Parse()).on("entry", (entry) => console.log(entry.path));
});
I get result: invalid signature: 0x6d74683c
I recompressed your image file into mine.zip and ran the code again:
const fs = require("fs");
const unzip = require("unzipper");
let inStream = fs.createReadStream("./mine.zip");`
inStream.on("open", ()=> {
inStream.pipe(unzip.Parse()).on("entry", (entry) => console.log(entry.path));
});
I get result: b7c4d01c3200558d594134f3a320c5.jpeg
mine.zip
We tried this image in multiple zip files (compressed on Ubuntu) and we get unexpected end of file especially if there are other images following this one, in debug I got invalid signature like you did, the main problem is that there is no error handling that enables me to skip this file and continue to the next one if the file inside the zip has invalid signature.
After checking it seems that with your zip I don't encounter the bug even if I add files to it but if I decompress it and compress it again in Ubuntu the combination of this specific file and the compression generates the end of file error, I still would like to have the ability to skip it because there is no issue with the other files -> I put it in a 250k images zip and I can't finish the extract but if I remove it from the zip I can, maybe it's a rare combination but still I can see that the end of file error appears in this library issues not once and would be nice to be able to skip files with bad signature
I'm encountering this issue as well. It would be really nice to be able to skip over a failed file.
I am having same issue. After digging (zip within 5k files), i found that is exactly this image (see attach). Ubuntu unzip works fine.
Error: unexpected end of file
at Zlib.zlibOnError [as onerror] (zlib.js:182:17)
at Zlib.callbackTrampoline (internal/async_hooks.js:131:14)
Error: Process exited with code 16
at process.<anonymous> (/app/node_modules/@google-cloud/functions-framework/build/src/invoker.js:96:22)
at process.emit (events.js:315:20)
at process.EventEmitter.emit (domain.js:467:12)
at process.exit (internal/process/per_thread.js:169:15)
at sendCrashResponse (/app/node_modules/@google-cloud/functions-framework/build/src/logger.js:38:9)
at process.<anonymous> (/app/node_modules/@google-cloud/functions-framework/build/src/invoker.js:92:44)
at process.emit (events.js:315:20)
at process.EventEmitter.emit (domain.js:467:12)
at eventToObjectGenerator.promiseCreated (/app/node_modules/bluebird/js/release/debuggability.js:186:33)
at activeFireEvent (/app/node_modules/bluebird/js/release/debuggability.js:229:44)
at fireRejectionEvent (/app/node_modules/bluebird/js/release/debuggability.js:614:14)
at Promise._notifyUnhandledRejection (/app/node_modules/bluebird/js/release/debuggability.js:61:9)
at Async._drainQueue (/app/node_modules/bluebird/js/release/async.js:138:12)
at Async._drainQueues (/app/node_modules/bluebird/js/release/async.js:146:10)
at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/bluebird/js/release/async.js:17:14)
at processImmediate (internal/timers.js:461:21)
Ubuntu 22.04 Node.js 14.15.4 It aslo failing in Google cloud Function (nodejs 14)
This problem appears when the zip file contains another zip file.
I am having the same error as well when compressing some a lot of files on Ubuntu, is there any solution for it?
Looks like this is happening when there is a zip inside zip
Another archive that causes this. It does not contain a zip. https://drive.google.com/file/d/1LExvQ6kdwJITJqgzWIfgOax_3s72FrD9/view?usp=share_link
I switched to https://github.com/cthackers/adm-zip
I am using this library to Parse a stream and extract from zip on the fly without storing the whole zip locally (stream to my server parse and send on entry to another server). I uploaded a zip with 250k images it fails on 'end of file Zlib zlibOnError 5 Z_BUF_ERROR' at the exact same entry every time, I removed this file and the parse works fine, I created a new zip with only that file and it fails every time (Using parse - for my code I must use parse since I don't store the whole file). I attached this file to the issue so you can check and see if there is a way to overcome this issue and perhaps skip it in flow, I tried to autodrain this specific entry but it mess up the next entry signature. p.s. I can open the file and view the image and also both in ubuntu and windows I can extract the file from the zip with no problem, the issue occurs only when I try to parse it from stream.
corrupted.zip