Closed evilpacket closed 7 years ago
i did a little more digging here and it looks like what's going on is the header for the package/rules
directory is being parsed with a size of 306. according to the tar internals documentation
On systems where disk allocation is performed on a directory basis, the size field will contain the maximum number of bytes (which may be rounded to the nearest disk block allocation unit) which the directory may hold. A size field of zero indicates no such limiting. Systems which do not support limiting in this manner should ignore the size field.
as a quick test, i patched headers.js
so that if (type === 'directory') size = 0
and the errors disappeared and the tarball extracted appropriately.
it sounds like for directories the size flag should probably only be informative for this module and an empty stream always returned. since we don't know what the end user is doing with the stream, we can't really enforce byte limits in an empty directory even if we wanted to.
i patched headers.js so that if (type === 'directory') size = 0 and the errors disappeared and the tarball extracted appropriately
Good catch :-) Could you do a pull-request?
it sounds like for directories the size flag should probably only be informative for this module and an empty stream always returned. since we don't know what the end user is doing with the stream, we can't really enforce byte limits in an empty directory even if we wanted to.
Absolutely, in fact I don't find any sense in having directories a size at all... seems it's mostly related to the size on disk as your quote indicate, similar to why ls
command show 4KB for directories on Ext4 filesystems, that on regular desktop Linux systems it's the default inode size.
i only suggested passing it along in an informational sense for the sake of being thorough. you never know if someone might want to use this module to write files to disk and is using a filesystem that supports these limits :)
i'll send over a pull request with a proposed fix shortly
I have a tarball from the registry (and others) that consistency fail with invalid tar header errors but are valid archive.
To replicate
wget https://registry.npmjs.org/eslint-config-metashop/-/eslint-config-metashop-1.5.0.tgz
Use the following test case
const gunzip = Gunzip(); const extract = Tar.extract(); const inputFile = Fs.createReadStream(process.argv[2]);
extract.on('error', function (err) { console.log(err); });
extract.on('entry', function (header, stream, callback) { stream.on('end', function () { return callback() }); stream.resume() });
inputFile.on('error', function (err) { console.log(err) });
inputFile.pipe(gunzip).pipe(extract);
Error: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped? at Object.exports.decode (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/headers.js:265:40) at Extract.onheader [as _onparse] (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:124:39) at Extract._write (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:248:8) at Extract._continue (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:212:28) at oncontinue (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:65:10) at Extract.onheader [as _onparse] (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:132:7) at Extract._write (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:248:8) at Extract._continue (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:212:28) at oncontinue (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:65:10) at Extract.onheader [as _onparse] (/Users/adam_baldwin/Documents/projects/tarHash/node_modules/tar-stream/extract.js:132:7)