npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.46k stars 3.15k forks source link

[BUG] npm install will randomly hang forever and cannot be closed when this occurs #4028

Open Metritutus opened 2 years ago

Metritutus commented 2 years ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

When running npm install it will sometimes hang at a random point. When it does this, it is stuck forever. CTRL+C will do nothing the first time that combination is pressed when this has occurred. Pressing that key combination the second time will make the current line (the one showing the little progress bar) disappear but that's it. No further responses to that key combination are observed.

The CMD (or Powershell) window cannot be closed regardless. The process cannot be killed by Task Manager either (Access Denied, although I'm an Administrator user so I'd assume the real reason is something non-permissions related). The only way I have found to close it is to reboot the machine.

My suspicion is it's some sort of deadlock, but this is a guess and I have no idea how to further investigate this. I've tried using Process Explorer to check for handles to files in the project directory from other processes but there are none. There are handles held by the Node process npm is using, and one for the CMD window hosting it, but that's it.

Even running with log-level silly yields no useful information. When it freezes there are no warnings or errors, it just sits on the line it was on. This is some log output from one of the times when it got stuck (I should again emphasise that the point where it gets stuck seems to be random, so the last line shown here isn't always the one it freezes on):

npm timing auditReport:init Completed in 49242ms
npm timing reify:audit Completed in 55729ms
npm timing reifyNode:node_modules/selenium-webdriver Completed in 54728ms
npm timing reifyNode:node_modules/regenerate-unicode-properties Completed in 55637ms
npm timing reifyNode:node_modules/ajv-formats/node_modules/ajv Completed in 56497ms
npm timing reifyNode:node_modules/@angular-devkit/schematics/node_modules/ajv Completed in 56472ms
[##################] \ reify:ajv: timing reifyNode:node_modules/@angular-devkit/schematics/node_modules/ajv Completed in 564

The only thing that I can think of right now is that Bit Defender (the only other application running) is interfering somehow, however it's the one application I can't turn off.

I've seen this issue occur on different projects, on different network and internet connections, and on different machines. Does anyone have any advice on how to investigate this, or at the very least a way to kill the process when it hangs like this without having to reboot the machine? Being forced to reboot when this issue occurs is perhaps the most frustrating thing in all of this.

Expected Behavior

npm install should either succeed or show an error. If it gets stuck it should either time-out or be closable by the user.

Steps To Reproduce

  1. Clear down the node_modules folder (ie with something like rmdir /q /s)
  2. Run. npm install
  3. Watch and wait.
  4. If it succeeds, repeat the above steps until the freeze is observed.

Environment

prefix = "C:\Users\\AppData\Roaming\npm"

; "user" config from C:\Users\.npmrc

//pkgs.dev.azure.com//_packaging//npm/registry/:_authToken = (protected)

; node bin location = C:\Program Files\nodejs\node.exe ; cwd = C:\Users\ ; HOME = C:\Users\ ; Run npm config ls -l to show all defaults.

PhilETaylor commented 1 year ago

interesting

0 verbose cli /opt/homebrew/Cellar/node/19.8.1/bin/node /opt/homebrew/bin/npm
1 info using npm@9.6.2
2 info using node@v19.8.1
3 timing npm:load:whichnode Completed in 0ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:/opt/homebrew/lib/node_modules/npm/npmrc Completed in 1ms
6 timing config:load:builtin Completed in 1ms
7 timing config:load:cli Completed in 1ms
8 timing config:load:env Completed in 0ms
9 timing config:load:file:/Users/phil/Sites/manage.mysites.guru/.npmrc Completed in 0ms
10 timing config:load:project Completed in 1ms
11 timing config:load:file:/Users/phil/.npmrc Completed in 0ms
12 timing config:load:user Completed in 0ms
13 timing config:load:file:/opt/homebrew/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:setEnvs Completed in 0ms
16 timing config:load Completed in 4ms
17 timing npm:load:configload Completed in 4ms
18 timing npm:load:mkdirpcache Completed in 0ms
19 timing npm:load:mkdirplogs Completed in 0ms
20 verbose title npm update
21 verbose argv "update" "--loglevel" "silly"
22 timing npm:load:setTitle Completed in 2ms
23 timing config:load:flatten Completed in 1ms
24 timing npm:load:display Completed in 2ms
25 verbose logfile logs-max:10 dir:/Users/phil/.npm/_logs/2023-03-28T12_29_56_696Z-
26 verbose logfile /Users/phil/.npm/_logs/2023-03-28T12_29_56_696Z-debug-0.log
27 timing npm:load:logFile Completed in 2ms
28 timing npm:load:timers Completed in 0ms
29 timing npm:load:configScope Completed in 0ms
30 timing npm:load Completed in 11ms
31 timing arborist:ctor Completed in 0ms
32 silly logfile start cleaning logs, removing 1 files
33 silly logfile done cleaning log files
34 timing idealTree:init Completed in 4ms
35 timing idealTree:userRequests Completed in 0ms
36 silly idealTree buildDeps
37 silly fetch manifest @fortawesome/fontawesome-pro@^6
38 verbose shrinkwrap failed to load node_modules/.package-lock.json out of date, updated: node_modules
[Hang]
npm ERR! code ETIMEDOUT
npm ERR! syscall connect
npm ERR! errno ETIMEDOUT
npm ERR! network request to https://npm.fontawesome.com/@fortawesome%2ffontawesome-pro failed, reason: connect ETIMEDOUT 2600:9000:2111:5600:c:7af5:a1c0:93a1:443
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.

Then if I remove "@fortawesome/fontawesome-pro": "^6", from package.json and try again... NO HANG and it completes!

Which when googling - ironically brings me to https://github.com/FortAwesome/Font-Awesome/issues/14592 in which I commented the same ipv6 issues in 2019....and which is still unresolved...

pwagsc commented 1 year ago

This may or may not be helpful to others but here is my final resolution. As Phil posted, I also through many trial and errors hung up on font-awesome-pro, and at times other items all depending on the state of the cache, etc. I finally dug into Ubuntu and discovered that all my various incarnations of "hangs" seemed to be the result of a freeway pileup - aka short memory page lock. On my failing system, memory is in short supply due to a 1Gb memory on a server running nginx and myql. Mysql is especially good at grabbin almost half a gig of memory when it spins up. Then when npm starts, linux gives up a bunch of its cache setaside to meet the voracious appetite of npm/nodejs thus exacerbating the performance environment. I can then watch the page lock pile up over 5 secs or so until its at 100% cpu.

Stopped mysql service and reran = no problems. Added a gig to the server (now 2G) - reran all failing scenarios - all ran without a hitch and very fast including the npm run dev I use in my Laravel env which had been running at a snails pace.

So that was my simple solution. I was mislead by all the intermittent scenarios that seemed to point to npm/nodejs - their memory management is pretty non existent so they are not blameless, but..

A quick look at htop showed only 62MB free (200+k available) before trying to run npm. Should have been a red flag for page swap pileups.

BTW - Ubunto single processor server with 2G is running mysql, nginx, app with IPv6 enabled. When npm install fires up it completes in 29 seconds. About 3 times slower if I clear cache which forces the load of 896 packages. But once again no problems at all with the additional memory.

2nd BTW - fontawesome-pro - be sure your license parameters are set in npm first. Also, I found the load was sometimes slow but always completed reasonably quickly with the additional memory. It was a major cause of failures before the memory fix - as in it always hung up.

Hope this helps someone else..

PhilETaylor commented 1 year ago

memory management

Not my issue, Im on a Mac Studio ultra with 128Gb RAM ;-)

brent-moffit commented 1 year ago

Not that it helps but am still blocked by this issue in npm > v6 on macOS 13.3. I tried disabling IPv6, clearing package files and cache (package-lock.json, node_modules, and npm cache clear -f), maxing out available memory, and even letting the process run over night (it consumed up to 4 GB memory and fully utilized a CPU core without getting unstuck).

The only thing that has unblocked me is downgrading npm to v6 (npm install -g npm@6). If there is any additional troubleshooting I can do to help folks in this thread please let me know.

Rafael-Guilherme commented 1 year ago

Same error here, with npm 9.6.5, only downgrading to v6 worked (npm install -g npm@6).

XavierDupessey commented 1 year ago

Disabling ip v6 works for me! Thanks :D

erikarens commented 1 year ago

Like @PhilETaylor mentioned, disabling iPv6 works for me. I'm on a MacBook Pro M2. I don't get why this is happening 🤔

I have installed nvm with brew and then node 18.6.0 / npm 9.5.1 with nvm. It worked fine for the first 5 days and then the problem occurred.

Thallius commented 1 year ago

Disabling ipv6 worked for me too on MAC M2

sudo networksetup -setv6off Wi-Fi

LKSFerreira commented 1 year ago

I tried everything. The number of times I uninstalled and installed node is over 8000, including completely cleaning my SSD with a hard reset and reinstalling Windows, everything from scratch, but nothing worked until the colleagues above mentioned something about disabling IPv6, and guess what, overnight NPM and NODE "said, hmm you know what I don’t like the look of this IPv6 anymore, I want to see who will make me work", just like that:

Congratulations to whoever covered up this workaround. If I had been stuck for one more day, I would have gone back to working as an Uber driver.

Community I love you guys.

image

orkylish commented 1 year ago

Disabling ipv6 worked for me too on MAC M2

sudo networksetup -setv6off Wi-Fi

OMG thanks to you and the rest above for suggesting disabling ipv6 -- I'm on mac M1 and this fixed the hangs for me

ScOut3R commented 1 year ago

npm config set strict-ssl false

worked for me

This is a security risk, I strongly discourage anyone from disabling strict SSL checking.

brdacost commented 1 year ago

Disabling ipv6 worked for me too on MAC M2

sudo networksetup -setv6off Wi-Fi

Thanks a lot!! This worked for me as well, and I'm using a MacOS with an Intell processor, so the error might not be related to the machine itself but to the npm version for some reason. 🤷

Artur- commented 1 year ago

Seems more like this is an issue with the npm registry and not the npm cli as

curl -6 https://registry.npmjs.org/vanilla-colorful --verbose

fails with

*   Trying [2606:4700::6810:1823]:443...
* connect to 2606:4700::6810:1823 port 443 failed: Operation timed out

on a mac M1

anandncode commented 1 year ago

@Artur-

The problem I face is, when the registry is not reachable, npm doesn't fail, it just hangs without any message. Though I haven't tested with the recent version of npm

kcjonesevans commented 1 year ago

in case this helps anyone looking, bc removing ipv6 is not a solution, though using a legacy version of npm might not be a solution for you either. Verified that this worked with node v16 and v18

npm install npm@8.11.0 -g
npm cache clear -f
npm i
mendezcode commented 1 year ago

✅ I was able to fix the hanging issue by installing npm@9 globally and running npm install again.

The project required node v14, which comes with npm v6, but using v9 worked just fine installing packages against v14.

cangSDARM commented 1 year ago

There is a stackoverflow question: https://stackoverflow.com/questions/73750687/what-do-idealtree-and-reify-mean-in-the-context-of-npm That explain why npm always hang up: because it reverify the whole huge "lock file vertal tree" over and over again! for every single freaking packge! so fuck sake, do not save lock file for any reason

StormJose commented 1 year ago
rm -Rf node_modules
rm package-lock.json
npm install --package-lock-only

Now its hanging

(⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂) ⠇ idealTree:[redacted-current-folder]: sill idealTree buildDeps

still hanging... still hanging... still hanging... still hanging...

give up...

ctrl+c disable ipv6...

npm install --package-lock-only
up to date, audited 667 packages in 44s
72 packages are looking for funding
npm install --no-optional
added 665 packages, and audited 666 packages in 5s

Worked to be as well

fengyufuchen commented 1 year ago

image ![Uploading image.png…]()

Any progress on this question?

smittynewton commented 1 year ago

I removed snapd from ubuntu and rebooted then it works. sudo apt remove snapd. Checked the process usage with top command and saw snapd was using 99% of cpu after trying to install node modules.

relign commented 1 year ago

Works on MacOS if I set IPV6 to Link-local (from Automatically). Thanks for the help. I found this solution in #3359

cwarren commented 1 year ago

I was running into this trying to get the amplify cli installed via windows powershell. Turning off ipv6 did not work, at least not on it's own. I then downgraded npm to v6 (npm install -g npm@6) and that did work (I still had ipv6 turned off as well).

Good luck out there!

PolarisPyra commented 1 year ago

Just wanted to confirm that running npm install -g npm@6 solved the issue for me as well... I have no idea why though!

mmiretsky commented 1 year ago

@cwarren, @PolarisPyra - does it work for you guys with npm 10?

cwarren commented 1 year ago

Nope - npm 10 did not work for me

Florin-Popescu commented 1 year ago

Doesn't work on steam deck. Not if disabling ipv6, not if rolling back to npm 6

wellrundigital commented 10 months ago

Ran into a similar problem with npm v10.2 and node v20.10. Installing locally helped (omitted the -g modifier).

AniJadeja commented 9 months ago

I know this is an old issue. But i was having the same issue and stumbled across this. So, I am mentioning a summary. If good old restart doesn't work then ,

  1. Try these steps mentioned by @kcjonesevans. Most of the time for some weird issues, caches can create new issues. So try clearing it.

in case this helps anyone looking, bc removing ipv6 is not a solution, though using a legacy version of npm might not be a solution for you either. Verified that this worked with node v16 and v18

npm install npm@8.11.0 -g
npm cache clear -f
npm i
  1. Try clearing cache force fully and setting up the registry again.

npm cache clear --force npm config set registry http://registry.npmjs.org

  1. Try reinstalling the node from scratch. These instruction could help.

    https://stackoverflow.com/questions/20711240/how-to-completely-remove-node-js-from-windows

  2. If none of these works then try yarn.

 npm install yarn
 yarn install

Hope this helps.

UPDATE

It is quite possible that the project is using some library and that library has problems. Try to create a new project and then run npm i again. In my case, I thought I may have broken something in powershell, however when I created a new project, npm i is running fine.

When I went back to my initial project, it still had problems with npm i. But this time I waited for a while, and ultimately it did finish installing the libraries.

I can't come to a conclusion, as the same repo was having problems in local machine but then on codespaces it worked like a charm. At the end, I would suggest anyone who is facing the same problem to use codespace for temporary development solution.

mirek commented 9 months ago

I suspect the issues I've seen may be related to mixing pnpm with npm - ie. if you use both, in some cases it may be impacting npm. I don't have any proof to back it up.

AniJadeja commented 9 months ago

That could be the case. But in my case, It all started wit npm ci. I ran npm ci and then suddenly the powershell window is getting frozen whenever i run npm i or npm ci. I recently had some changes in windows OS, ie. fixing broken services. But, i suspect that I may have broken something else which is preventing npm install commands.

in my case yarn is working fine. When I ran the commands on codepaces with the same repo to identify if I actually somehow changed package files which are causing some library issues, codespace ran just fine. so I must have mistakenly broken some service(s) related to the powershell not npm. Because npm is running fine on codespace.

I am preparing for a soft reset as of now. So, I think that for my case, i know the issue.

Riekr commented 9 months ago

I tried with npm install -g npm@6 and worked but I noticed that while installing angular I've got the prompt

Would you like to share anonymous usage data with the Angular Team at Google...

I cancelled the job and reinstalled 10.4.0 but set export NG_CLI_ANALYTICS=ci (as pointed here), cleaned _nodemodules and did a cache verify... it worked again.

May it be that versions following npm@6 broke stdin handling?

flensrocker commented 9 months ago

If I have to guess, I would say this is a ipv4/ipv6 problem. If the connection uses ipv4 it works, and if it picks ipv6 it doesn't. And it's random.

simonweikl commented 9 months ago

I had the same issue. For me deleting the ".npmrc" fixed it

AbhijithKizhakkepat commented 8 months ago

I had the same issue. I followed the following steps to fix the issue:

The following versions are compatible and I don't see npm hanging issue: node v18.18.2 and npm v10.2.4 Hope this is helpful.

leonp5 commented 8 months ago

I have the same issue quitting works when i press CTRL + C twice. node 20.11.0 npm 10.4 Docker 25.0.1 in a docker container on a Linux machine until now i didn't find a solution and nothing from the proposals above worked.

matkoniecz commented 8 months ago

In my case np install was failing before npm cache clear -f and worked afterwards. Note that I am not entirely sure was it coincidence that it started working after this, or is it a viable workaround.

jinbe0 commented 7 months ago

Still not working on slow network or high-latency network such as bulletproof VPN or Tor. There is no workaround except installing on fast network environment and create tar ball of node_modules, then copy to local and extract it. Please test on slow network + no IPv6 before next release.

danaYatsuta commented 7 months ago

NodeJS version: 21.7.0 npm version: 10.5.0

npm install hangs at npm sill reify moves {} for a very long time before continuing, probably around 10 minutes. Downgrading npm to 8.11.0 solves the issue.

pavelloz commented 7 months ago

FWIW I've found this issue today because of having exact same issue (node 20.11.1, npm 10.2.4), this is what fixed it for me:

mayhew3 commented 7 months ago

Same issue for me. Was running Node v20.12.0, npm 10.5.0. Reinstalling didn't work, removing node_modules didn't work, clearing the cache didn't work. Only thing that worked was downgrading to Node v18.20.0, npm 8.11.0.

heiliuer commented 6 months ago

I have a same problem! Finaly i find some host of 'package.lock.json' resovle url can not be access(ssl timeout), replace it soveled.

rharkor commented 6 months ago

Same problem here..

sameerfa commented 6 months ago

@batressc bro! this worked for me. i tried everything else and decided to also give this a shot as much as this sounds unconventional. but it worked! thanks a ton! 🙏🏼

zharvey-wae commented 6 months ago

Same problem for us, downgrading to Node 18 solved the problem, that's a bummer. Always a fail when downgrading is the solution :-/

yonirab commented 5 months ago

I had the same horrible problem with node 21.7.0, npm 10.5.0. Upgrading to node 22.1.0, npm 10.7.0 fixed it for me.

SkullRex001 commented 5 months ago

just downgrade npm : npm install -g npm@8

ljharb commented 5 months ago

npm < 10 is unsupported, and downgrading to an unsupported npm is not a wise workaround.

bananabrann commented 5 months ago

npm installs only fail for me in an Ubuntu 20.04 container and in WSL2 (also Ubuntu 20.04). I have not tried downgrading to npm@8, but my issue seems to be something with the HTTP requests

Everything works like a charm on Node 18 and npm@10 if I disable IPv6 for the session

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

I didn't look further to tell if this is an npm issue or not, or whether it's an Ubuntu/WSL2/Grub issue

SkullRex001 commented 5 months ago

yep , same problem. I am also using Ubuntu 20.04 container and in WSL2. But for now , downgrading to npm@8 does the trick for me.

danielrosehill commented 5 months ago

am on ubuntu and have this problem constantly (at least in vs code)