Incorrect header check in 0.16.1

[ocombe]

Hello,

since the last update (0.16.1) and the change to the compression I get a lot of errors when running shrinkpack:

↓ @types/core-js@0.9.29 from https://registry.npmjs.org/@types/core-js/-/core-js-0.9.29.tgz
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: incorrect header check
    at Zlib._handle.onerror (zlib.js:370:17)

↓ @types/jasmine@2.2.31 from https://registry.npmjs.org/@types/jasmine/-/jasmine-2.2.31.tgz
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: incorrect header check
    at Zlib._handle.onerror (zlib.js:370:17)

Some packages are fine, but a lot of them fail (maybe 1/10th)...

I cleaned up my node_modules and npm cache, removed the npm-shrinkwrap.json file, reinstalled the node modules and tried again, same problem...

I tried the --compress option and everything runs fine, which means that it's definitely the problem here

I'm on windows 10 x64, using node 6.3.0 and npm 3.10.6

edit: I'm wondering if it has to do with the namespace npm modules like @types/... or @angular/...

[JamieMason]

Thanks @ocombe, managed to reproduce it using the below and am looking into it now;

{
  "name": "test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@types/core-js": "0.9.29",
    "@types/jasmine": "2.2.31"
  }
}

[ocombe]

Awesome, thanks :)

[JamieMason]

I think the problem is that copyFile() is not rate-limited like the other fs or child_process jobs are. I've done that locally and it seems to work, I'll create a release now.

[JamieMason]

Waiting for build #14 of shrinkpack.

[ocombe]

It looks like it passed :)

[JamieMason]

Released in 0.16.2, thanks @ocombe for reporting.

[ocombe]

It worked almost until the end where I got:

↓ xmlhttprequest@1.8.0 from https://registry.npmjs.org/xmlhttprequest/-/xmlhttprequest-1.8.0.tgz
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: incorrect header check
    at Zlib._handle.onerror (zlib.js:370:17)

:(

[ocombe]

Sure, here it is:

{
  "name": "app",
  "version": "0.0.0",
  "license": "MIT",
  "engines": {
    "npm": "<3.8.8 >=3.10.4"
  },
  "scripts": {
    "build": "webpack --progress --profile --bail",
    "local": "webpack-dev-server --progress --inline --port=8080 --watch",
    "shrink": "npm prune && npm shrinkwrap --dev && shrinkpack --compress",
    "test": "karma start",
    "test-watch": "karma start --no-single-run --auto-watch",
    "start": "webpack-dev-server --progress --inline --hot --https --port=9001 --host=0.0.0.0 --watch",
    "docker": "webpack-dev-server --progress --inline --hot --https --port=9001 --host=0.0.0.0 --watch",
    "typings": "typings install",
    "watch": "webpack --watch --progress --profile --hot --watch-poll"
  },
  "dependencies": {
    "@angular/common": "2.0.0-rc.5",
    "@angular/compiler": "2.0.0-rc.5",
    "@angular/core": "2.0.0-rc.5",
    "@angular/forms": "0.3.0",
    "@angular/http": "2.0.0-rc.5",
    "@angular/platform-browser": "2.0.0-rc.5",
    "@angular/platform-browser-dynamic": "2.0.0-rc.5",
    "@angular/router": "3.0.0-rc.1",
    "@ngrx/core": "1.0.1",
    "@ngrx/db": "1.1.1",
    "@ngrx/store": "2.0.1",
    "bootstrap": "4.0.0-alpha.3",
    "core-js": "2.4.1",
    "d3": "4.2.1",
    "d3-selection-multi": "1.0.0",
    "flatpickr": "1.9.1",
    "lodash": "4.14.2",
    "moment": "2.14.1",
    "ng2-translate": "2.2.2",
    "rxjs": "5.0.0-beta.6",
    "zone.js": "0.6.12"
  },
  "devDependencies": {
    "@types/core-js": "0.9.29",
    "@types/d3-array": "0.0.1",
    "@types/d3-axis": "0.0.1",
    "@types/d3-color": "0.0.1",
    "@types/d3-force": "0.0.1",
    "@types/d3-format": "0.0.1",
    "@types/d3-interpolate": "0.0.1",
    "@types/d3-scale": "0.0.1",
    "@types/d3-selection": "0.0.1",
    "@types/d3-shape": "0.0.1",
    "@types/d3-time-format": "0.0.1",
    "@types/d3-transition": "0.0.1",
    "@types/d3-zoom": "0.0.1",
    "@types/hammerjs": "2.0.30",
    "@types/jasmine": "2.2.31",
    "@types/node": "6.0.32",
    "@types/protractor": "1.5.16",
    "@types/selenium-webdriver": "2.44.26",
    "@types/source-map": "0.1.26",
    "@types/uglify-js": "2.0.27",
    "@types/webpack": "1.12.31",
    "angular2-template-loader": "0.4.0",
    "autoprefixer": "6.4.0",
    "awesome-typescript-loader": "2.1.1",
    "clean-webpack-plugin": "0.1.10",
    "codelyzer": "0.0.26",
    "copy-webpack-plugin": "3.0.1",
    "css-loader": "0.23.1",
    "extract-text-webpack-plugin": "1.0.1",
    "file-loader": "0.9.0",
    "html-loader": "0.4.3",
    "html-webpack-plugin": "2.22.0",
    "json-loader": "0.5.4",
    "node-sass": "3.8.0",
    "null-loader": "0.1.1",
    "phantomjs-prebuilt": "2.1.11",
    "postcss-loader": "0.9.1",
    "raw-loader": "0.5.1",
    "sass-loader": "4.0.0",
    "shrinkpack": "0.16.1",
    "style-loader": "0.13.1",
    "stylelint": "7.1.0",
    "stylelint-scss": "1.3.4",
    "stylelint-webpack-plugin": "0.3.0",
    "ts-helpers": "1.1.1",
    "tslint": "3.14.0",
    "tslint-loader": "2.1.5",
    "typescript": "2.0.0",
    "url-loader": "0.5.7",
    "webpack": "1.13.1",
    "webpack-dev-server": "1.14.1",
    "webpack-notifier": "1.3.2"
  }
}

I use the npm command npm run shrink to start the process

[JamieMason]

Thanks, I'd been able to reproduce and think I've just found a fix. AFAICT those @types scoped packages aren't gzipped, so trying to decompress them results in errors. I'm using https://github.com/mafintosh/gunzip-maybe locally and this seems to fix the problem.

I'll keep testing.

[ocombe]

That makes sense :)

[JamieMason]

Running the build now, if you could try it out ahead of release by running;

npm install -g https://github.com/JamieMason/shrinkpack.git#8d3836e56b64010baf9dc9fa859ed2df8d4c00c7

That'd be a great help, thanks.

[ocombe]

It took like forever but it worked ! My folder is now taking 131mo instead of 20mo :D

[JamieMason]

My folder is now taking 131mo instead of 20mo :D

Just checking, are you saying here thatnode_shrinkwrap is 131mb in size instead of 20mb? If so, you can use shrinkpack --compress to continue using .tgz files. They'll be much smaller but less suited to Git (which works best with text files).

[JamieMason]

Released in 0.16.3 👍

[ocombe]

Yes the folder is a lot bigger now (which is normal since the archives ain't compressed anymore). I think I'll keep using the compress option for now, we'll see if I have problems with git later, but for now I haven't.

[JamieMason]

sounds good, thanks @ocombe