search

thejoshwolfe / yazl

yet another zip library for node
MIT License
341 stars 45 forks source link

Error: file data stream has unexpected number of bytes #32

Closed ariporad closed 7 years ago

ariporad commented 7 years ago

Hi!

For some reason, I occasionally get the following error when trying to create a zip file. I'd be really grateful if you were able to provide any insight.

Uncaught Exception:
Error: file data stream has unexpected number of bytes
    at ByteCounter.<anonymous> (/Applications/Castbridge.app/Contents/Resources/app.asar/node_modules/yazl/index.js:144:99)
    at emitNone (events.js:91:20)
    at ByteCounter.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Thanks!

thejoshwolfe commented 7 years ago

My best guess is that the file you're trying to add to the archive is being modified while yazl is attempting to read it. The specific error means that the fs.ReadStream that's reading the file produced a different number of bytes than the fs.stat() call predicted it would. (Either that or there's a serious bug in yazl.)

If I had to guess, I would say that you're writing files to the file system from some source and then immediately zipping them up into an archive. There's a really easy mistake to make with Node.js programming where you don't wait for a WriteStream to fully finish writing its data before you try to read the file. For example, if you wait for a stream earlier in the pipeline to end instead, then you'll get intermittent failures like the one you're getting.

That's all a really specific guess though. Your situation may be different. Can you verify that the files you're sending to addFile() are not changing while yazl is reading them?

ariporad commented 7 years ago

Oh, that totally makes sense! Thanks!

rash805115 commented 6 years ago

Or it could be a zero byte file. At lease for me that was the case.

thejoshwolfe commented 6 years ago

zero byte files should not cause this error. @rash805115 can you open an issue and describe the problem you're experiencing?

rash805115 commented 6 years ago

@thejoshwolfe I tested again. It turns out I also had the same issue of reading and writing from same location. I was reading before the write finished, hence thought it was a zero byte error. Thanks for your help. Let me know if I need to delete my first comment so that folks don't get confused.

thejoshwolfe commented 6 years ago

that's a relief to hear.

(no need to delete any comments.)