search

mafintosh / tar-stream

tar-stream is a streaming tar parser and generator.
MIT License
400 stars 93 forks source link

Error: Writable stream closed prematurely #154

Closed catdevnull closed 1 year ago

catdevnull commented 1 year ago

Since 3.1, I get this error with my code:

node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: Writable stream closed prematurely
    at Pipeline.done (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:434:43)
    at WritableState.afterDestroy (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:494:43)
    at Sink._destroy (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.2/node_modules/tar-stream/pack.js:102:5)
    at WritableState.updateNonPrimary (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:201:16)
    at WritableState.update (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:183:72)
    at WritableState.updateWriteNT (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:532:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Emitted 'error' event on Source instance at:
    at ReadableState.afterDestroy (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:483:19)
    at Source._destroy (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.2/node_modules/tar-stream/extract.js:98:5)
    at ReadableState.updateNonPrimary (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:376:16)
    at ReadableState.update (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:357:71)
    at ReadableState.updateReadNT (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:525:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

Node.js v18.16.0

This is code that downloads container images, by downloading each layer, parsing every layer and creating a new single tar stream to pass to mksquashfs to create an image with everything inside the container.

Full repro is in https://gitea.nulo.in/Nulo/fireactions/src/branch/tar-stream-repro (branch tar-stream-repro)

To execute, do:

cd js
pnpm install
esbuild container-baby.ts > container-baby.js
node container-baby.js

This will download a tiny OCI image's layers and try to do this layers -> tar -> mksquashfs pipeline. It fails with the error above in tar-stream@3.1 but succeeds in @3.0.

The tar-stream logic is in tar2squashfs.js.

Sorry for the messy repro!

mafintosh commented 1 year ago

Thanks! Should be fixed in 3.1.3