shrinkpack command error failed to download @angular/platform-browser@2.0.0-rc.4

yaroslavya commented 8 years ago

If reporting a bug, please use the following template.

Expected behaviour

shrinkpack should get the files from the node_modules folder and put them under the node_shrinkwrap folder

Actual behaviour

gives the following error:
! failed to download @angular/platform-browser@2.0.0-rc.4                       
! Please raise an issue at https://github.com/JamieMason/shrinkpack/issues      

    Error: Command failed: C:\Windows\system32\cmd.exe /s /c "npm cache --scope=
@angular add file:node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz"     
    npm ERR! addLocal Could not install C:\angular2-webpack\node_shr
inkwrap\@angular-platform-browser-2.0.0-rc.4.tgz                                
    npm ERR! Windows_NT 6.1.7601                                                
    npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\USERNAME\\A
ppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "cache" "--scope=@angu
lar" "add" "file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz"     
    npm ERR! node v4.2.1                                                        
    npm ERR! npm  v3.8.6                                                        
    npm ERR! path C:\angular2-webpack\node_shrinkwrap\@angular-platf
orm-browser-2.0.0-rc.4.tgz                                                      
    npm ERR! code ENOENT                                                        
    npm ERR! errno -4058                                                        
    npm ERR! syscall open                                                       

    npm ERR! enoent ENOENT: no such file or directory, open 'C:\angu
lar2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz'          
    npm ERR! enoent ENOENT: no such file or directory, open 'C:\angu
lar2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz'          
    npm ERR! enoent This is most likely not a problem with npm itself           
    npm ERR! enoent and is related to npm not being able to find a file.        
    npm ERR! enoent                                                             

    npm ERR! Please include the following file with any support request:        
    npm ERR!     C:\angular2-webpack\npm-debug.log                  

        at ChildProcess.exithandler (child_process.js:203:12)                   
        at emitTwo (events.js:87:13)                                            
        at ChildProcess.emit (events.js:172:7)                                  
        at maybeClose (internal/child_process.js:818:16)                        
        at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

Steps to reproduce

git clone the https://github.com/preboot/angular2-webpack
npm install
npm run shrinkwrap
shrinkpack

Software versions used

OS         : Windows 7
node.js    : v4.2.1
npm        : 3.8.6
shrinkpack : 0.13.1

Log output when running shrinkpack

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\iiakovlev\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'cache',
1 verbose cli   '--scope=@angular',
1 verbose cli   'add',
1 verbose cli   'file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz' ]
2 info using npm@3.8.6
3 info using node@v4.2.1
4 silly cache add args [ 'file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz' ]
5 verbose cache add spec file:node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz
6 silly cache add parsed spec Result {
6 silly cache add   raw: 'file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz',
6 silly cache add   scope: null,
6 silly cache add   name: null,
6 silly cache add   rawSpec: 'file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz',
6 silly cache add   spec: 'C:\\Programming\\angular2-webpack\\node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz',
6 silly cache add   type: 'local' }
7 error addLocal Could not install C:\Programming\angular2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz
8 verbose stack Error: ENOENT: no such file or directory, open 'C:\Programming\angular2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz'
8 verbose stack     at Error (native)
9 verbose cwd C:\Programming\angular2-webpack
10 error Windows_NT 6.1.7601
11 error argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\iiakovlev\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "cache" "--scope=@angular" "add" "file:node_shrinkwrap\\@angular-platform-browser-2.0.0-rc.4.tgz"
12 error node v4.2.1
13 error npm  v3.8.6
14 error path C:\Programming\angular2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz
15 error code ENOENT
16 error errno -4058
17 error syscall open
18 error enoent ENOENT: no such file or directory, open 'C:\Programming\angular2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz'
19 error enoent ENOENT: no such file or directory, open 'C:\Programming\angular2-webpack\node_shrinkwrap\@angular-platform-browser-2.0.0-rc.4.tgz'
19 error enoent This is most likely not a problem with npm itself
19 error enoent and is related to npm not being able to find a file.
20 verbose exit [ -4058, true ]

Contents of your npm-shrinkwrap.json or package.json

yaroslavya-shrinkwrap.json.txt

JamieMason commented 8 years ago

I'm away at the moment but @5id I think this might be your slice on @ for scoped packages?

yaroslavya commented 8 years ago

One more detail: I tried it several times, by removing the repo and starting from scratch and it failed every time. But it fails on different @angular packages. So its not @angular/platform-browser all the time. Hope it helps :8ball:

JamieMason commented 8 years ago

I thought this might just be to do with scoped packages, but the following worked;

$ mkdir test
$ cd test
$ npm init -y
$ cat package.json

    {
      "name": "test",
      "version": "1.0.0",
      "dependencies": {
        "@angular/common": "2.0.0-rc.1",
        "@angular/compiler": "2.0.0-rc.1",
        "@angular/core": "2.0.0-rc.1",
        "@angular/http": "2.0.0-rc.1",
        "@angular/platform-browser": "2.0.0-rc.1",
        "@angular/platform-browser-dynamic": "2.0.0-rc.1",
        "@angular/router": "2.0.0-rc.1",
        "rxjs": "5.0.0-beta.6",
        "zone.js": "0.6.12"
      },
      "keywords": [],
      "license": "ISC",
      "main": "index.js"
    }

$ npm install
$ npm shrinkwrap --dev
$ shrinkpack

    i 9 dependencies in npm-shrinkwrap.json
    i 0 need removing from ./node_shrinkwrap
    i 9 need adding to ./node_shrinkwrap
    i 2 are in your npm cache
    i 7 need downloading
    i 0 have a missing "resolved" property
    ↓ @angular/common@2.0.0-rc.1 from http://registry.npmjs.org/@angular/common/-/common-2.0.0-rc.1.tgz
    ↓ @angular/compiler@2.0.0-rc.1 from http://registry.npmjs.org/@angular/compiler/-/compiler-2.0.0-rc.1.tgz
    ↓ @angular/core@2.0.0-rc.1 from http://registry.npmjs.org/@angular/core/-/core-2.0.0-rc.1.tgz
    ↓ @angular/http@2.0.0-rc.1 from http://registry.npmjs.org/@angular/http/-/http-2.0.0-rc.1.tgz
    ↓ @angular/platform-browser@2.0.0-rc.1 from http://registry.npmjs.org/@angular/platform-browser/-/platform-browser-2.0.0-rc.1.tgz
    ↓ @angular/platform-browser-dynamic@2.0.0-rc.1 from http://registry.npmjs.org/@angular/platform-browser-dynamic/-/platform-browser-dynamic-2.0.0-rc.1.tgz
    ↓ @angular/router@2.0.0-rc.1 from http://registry.npmjs.org/@angular/router/-/router-2.0.0-rc.1.tgz
    + @angular/platform-browser-dynamic@2.0.0-rc.1
    + zone.js@0.6.12
    + @angular/http@2.0.0-rc.1
    + @angular/router@2.0.0-rc.1
    + @angular/platform-browser@2.0.0-rc.1
    + @angular/common@2.0.0-rc.1
    + @angular/core@2.0.0-rc.1
    + @angular/compiler@2.0.0-rc.1
    + rxjs@5.0.0-beta.6
    shrinkpack +9 -0 ↓7 ✓0 00:02

$ rm -rf node_modules
$ npm install

I will try to test the same on Windows.

Other things of note

Some of the entries in your shrinkwrap have empty paths, such as file:node_shrinkwrap\\abbrev-1.0.9.tgz.
The paths are \ rather than / – I wonder if npm honours those in a shrinkwrap.

JamieMason commented 8 years ago

Hey @yaroslavya, I ran the following on a windows vm and all worked fine, however, I just noticed your version of npm and I think that is the problem – it needs to be 3.10.4 or newer.

git clone https://github.com/preboot/angular2-webpack.git
cd angular2-webpack
npm install 
npm shrinkwrap --dev
shrinkpack
rmdir /s /q node_modules/
npm install

Can you upgrade npm and see if that sorts it? Thanks.

jesstelford commented 8 years ago

I'm getting similar errors on npm@3.10.5 - am running on Ubuntu 14.04, with node@5.11.1.

Mine involves scoped packages also (in our case; private packages, so I can't provide a reproduction case), and it also errors on different packages each time it's run.

It's not something like a request timeout or something is it?

edit Same error on npm@3.10.6

edit 2. Some investigation reveals the following lines in my (and the OP's) npm-shrinkwrap.json:

    "@domain-group/fe-helper": {
      "version": "2.2.2",
      "from": "@domain-group/fe-helper@>=2.2.0 <3.0.0",
      "resolved": "http://registry.npmjs.org/@domain-group/fe-helper/-/fe-helper-2.2.2.tgz"
    },
    "@domain-group/fe-lint-scss": {
      "version": "1.0.2",
      "from": "@domain-group/fe-lint-scss@>=1.0.1 <2.0.0",
      "resolved": "file:node_shrinkwrap/@domain-group-fe-lint-scss-1.0.2.tgz"
    },

There are two different resolved field protocols:

http://registry.npmjs.org/@domain-group/fe-helper/-/fe-helper-2.2.2.tgz
file:node_shrinkwrap/@domain-group-fe-lint-scss-1.0.2.tgz

The interesting part is the node_shrinkwrap prefix - I'm not sure where this comes from as I am certain I don't have that local folder in my working directory.

edit 3:

Checking out the code, I can see it's attempting to run the following commands:

npm cache --scope=@domain-group add http://registry.npmjs.org/@domain-group/fe-helper/-/fe-helper-2.2.2.tgz
npm cache --scope=@domain-group add file:node_shrinkwrap/@domain-group-fe-lint-scss-1.0.2.tgz

But the folder node_shrinkwrap doesn't exist, so the command fails (causing the original error).

edit 4: To clarify, the command I'm running is;

rm -rf npm-shrinkwrap.json && rm -rf node_modules && rm -rf node_shrinkwrap && npm install && npm shrinkwrap

I'm going to re-run that again but prefixed with a npm cache clear to see if there's something funky going on there.

edit 5: A-ha! Running npm cache clear fixed it. Going to try get a reproducable case together now.

jesstelford commented 8 years ago

I can't seem to reproduce the exact case above (where some fields in npm-shrinwrap.json have a file:/ prefix), but this is very close:

git clone https://github.com/preboot/angular2-webpack.git
cd angular2-webpack
npm cache clear
npm install 
npm shrinkwrap
shrinkpack

This succeeds. Note that the URLs for "resolved" fields in the npm-shrinkwrap.json are all http:// urls pointing to npmjs.org.

Now, run:

rm -rf node_shrinkwrap # (1)
shrinkpack

This fails. Note that at point (1), the URLs for "resolved" fields in the npm-shrinkwrap.json now begin with ./node_shrinkwrap.

yaroslavya commented 8 years ago

Maybe it will shed some light, but my very first installation and shrinkpack was successful. Then I`ve deleted the node_modules folder and had this error I reported. I didnt get a chance to try it out yet. Will do this weekend. Maybe the updated npm will work just fine on windows. I will also try to clear the cache, maybe it does the trick.

mateatslc commented 8 years ago

Similar problem here, I'm running the following commands:

(1) rm -rf ./node_modules && rm -f npm-shrinkwrap.json && npm cache clear && npm i && npm prune (2) npm shrinkwrap --dev

fine and dandy till this point, then...

(3) rm -rf ./node_shrinkwrap && shrinkpack

here shrinkpack fails with this:

...
! failed to download aws4@1.4.1
! Please raise an issue at https://github.com/JamieMason/shrinkpack/issues

    Error: Command failed: npm cache add https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz
    npm ERR! fetch failed https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz
    npm ERR! fetch failed https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz
    npm ERR! fetch failed https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz
    npm ERR! Darwin 15.5.0
    npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "cache" "add" "https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz"
    npm ERR! node v6.1.0
    npm ERR! npm  v3.8.6
    npm ERR! code ENOTFOUND
    npm ERR! errno ENOTFOUND
    npm ERR! syscall getaddrinfo

    npm ERR! network getaddrinfo ENOTFOUND lunabuild.akamai.com lunabuild.akamai.com:443
...

this is on OSX 10.11.5, see node and npm version above. these exact commands worked without a hitch for months till last week.

JamieMason commented 8 years ago

Thanks everyone, let's focus on https://github.com/preboot/angular2-webpack.git as that's a public repo we can all access and reproduce this issue with.

tl;dr – I have no idea what the answer is yet and there is a lot of info in this issue. Here is a dump of what I know so far.

People are experiencing npm ERR! network getaddrinfo ENOTFOUND lunabuild.akamai.com lunabuild.akamai.com:443
https://lunabuild.akamai.com does not resolve when I put it into my web browser.
The issue can be reproduced using https://github.com/preboot/angular2-webpack.git.
npm install --loglevel http on that repo produces this output angular2-webpack-loglevel-http.txt.
There's no mention https://lunabuild.akamai.com in that install output.
npm shrinkwrap --dev on that repo produces npm-shrinkwrap.json.txt.
The shrinkwrap does contain references to https://lunabuild.akamai.com;

"aws4": {
  "version": "1.4.1",
  "from": "aws4@>=1.2.1 <2.0.0",
  "resolved": "https://lunabuild.akamai.com/nexus/content/groups/npm-all/aws4/-/aws4-1.4.1.tgz"
}

rm -rf node_modules and npm install --loglevel http to install from the npm-shrinkwrap.json still doesn't include references to https://lunabuild.akamai.com (not sure why??).
None of the above involved shrinkpack at all.
npm doesn't seem to be using all of the entries in npm-shrinkwrap.json, otherwise why wouldn't https://lunabuild.akamai.com ever be getting hit?
shrinkpack does use all of the entries in npm-shrinkwrap.json if;
- the package is not in node_shrinkwrap.
- and the package is not in ~/.npm.

Q: Why would npm not hit the URL defined in resolved for a dependency in npm-shrinkwrap.json?

jesstelford commented 8 years ago

@JamieMason I think the lunabuild.akamai.com errors are actually new and different.

See https://github.com/strongloop/fsevents/issues/141

I believe this is a red herring masking the original issue that myself and @yaroslavya had.

JamieMason commented 8 years ago

Thanks @jesstelford, let's avoid angular2-webpack.git then in that case.

Would someone mind creating a reduced test case which reproduces the issue please? I'm pretty lost and the OP expectation of "shrinkpack should get the files from the node_modules folder and put them under the node_shrinkwrap folder" is too broad in it's description to narrow down what's needed.

Apologies if I'm missing something obvious here.

mateatslc commented 8 years ago

My problem seems to be solved by https://github.com/strongloop/fsevents/issues/141#issuecomment-233320756 👍

JamieMason commented 8 years ago

I didnt get a chance to try it out yet. Will do this weekend. Maybe the updated npm will work just fine on windows. I will also try to clear the cache, maybe it does the trick. – @yaroslavya

Could you let me know how this goes please Yaroslav?

yaroslavya commented 8 years ago

@JamieMason Could not get to it yet. Will be able to check it out tomorrow and report the result here. Sorry for the delay.

JamieMason commented 8 years ago

No problem at all @yaroslavya

yaroslavya commented 8 years ago

The npm version 3.10.5 worked just fine with preboot project. Didn't try the previous versions. Just in case anyone else has this problem I have a Windows 8 (didnt try it on win7 on which I had the initial issue). The issue was reproduced in absolutely the same way on win8. So I think that the latest npm version fixes it, so you can close it.

JamieMason commented 8 years ago

Thanks for looking into this for us @yaroslavya, appreciate it.

JamieMason / shrinkpack