mafintosh / tar-stream

tar-stream is a streaming tar parser and generator.
MIT License
411 stars 93 forks source link

Error: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped? #158

Open catdevnull opened 1 year ago

catdevnull commented 1 year ago
    at Object.decode (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.4/node_modules/tar-stream/headers.js:116:43)
    at Extract._consumeHeader (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.4/node_modules/tar-stream/extract.js:144:30)
    at Extract._update (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.4/node_modules/tar-stream/extract.js:272:41)
    at Extract._write (/home/diablo/proy/fireactions/js/node_modules/.pnpm/tar-stream@3.1.4/node_modules/tar-stream/extract.js:287:10)
    at WritableState.update (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:180:16)
    at WritableState.updateWriteNT (/home/diablo/proy/fireactions/js/node_modules/.pnpm/streamx@2.15.0/node_modules/streamx/index.js:532:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

repro:

import gunzip from "gunzip-maybe";
import { Duplex, Writable } from "node:stream";
import { extract } from "tar-stream";

const ex = extract();

ex.on("entry", (header, stream, next) => {
  console.debug(header.name);
  stream.on("end", function () {
    next(); // ready for next entry
  });

  stream.resume();
});
const res = await fetch("https://0x0.st/HQqY.bin");
const stream = res.body;
await stream.pipeThrough(Duplex.toWeb(gunzip())).pipeTo(Writable.toWeb(ex));
mafintosh commented 1 year ago

Can you add a proper test with the tar that fails as a fixture, without the fetch and stream wrappers

catdevnull commented 1 year ago

I don't really know what's the issue so I can't develop a proper fixture for it, but here you go: https://gitea.nulo.in/Nulo/tar-stream (uploaded to a self-hosted repo because GitHub doesn't allow LFS in forks...)

personalizedrefrigerator commented 9 months ago

I'm encountering a similar (but maybe completely different?) issue.

I'm using tar-stream with feross/buffer in React Native (and a tar file generated with tar-stream).

For me, val.subarray(offset, end).toString() in decodeOct is evaluating to a list of numbers instead of as ASCII (e.g. 48,48,48,54,52,52 instead of 000644). https://github.com/mafintosh/tar-stream/blob/03da589543d45a4eeed561b4029e4b538e4fa990/headers.js#L307

Replacing that line with

return parseInt(Buffer.from(val).toString('utf-8', offset, end), 8)

and adding

const {Buffer} = require('buffer');

to the top of the file seems to fix the issue.

Edit: Upstream issue: https://github.com/feross/buffer/issues/329

FrederikBolding commented 8 months ago

I ran into a similar problem as @personalizedrefrigerator and ended up submitting a PR for it: https://github.com/mafintosh/tar-stream/pull/164