[BUG] npm install will randomly hang forever and cannot be closed when this occurs

Metritutus commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

This issue exists in the latest npm version

[X] I am using the latest npm

Current Behavior

When running npm install it will sometimes hang at a random point. When it does this, it is stuck forever. CTRL+C will do nothing the first time that combination is pressed when this has occurred. Pressing that key combination the second time will make the current line (the one showing the little progress bar) disappear but that's it. No further responses to that key combination are observed.

The CMD (or Powershell) window cannot be closed regardless. The process cannot be killed by Task Manager either (Access Denied, although I'm an Administrator user so I'd assume the real reason is something non-permissions related). The only way I have found to close it is to reboot the machine.

My suspicion is it's some sort of deadlock, but this is a guess and I have no idea how to further investigate this. I've tried using Process Explorer to check for handles to files in the project directory from other processes but there are none. There are handles held by the Node process npm is using, and one for the CMD window hosting it, but that's it.

Even running with log-level silly yields no useful information. When it freezes there are no warnings or errors, it just sits on the line it was on. This is some log output from one of the times when it got stuck (I should again emphasise that the point where it gets stuck seems to be random, so the last line shown here isn't always the one it freezes on):

npm timing auditReport:init Completed in 49242ms
npm timing reify:audit Completed in 55729ms
npm timing reifyNode:node_modules/selenium-webdriver Completed in 54728ms
npm timing reifyNode:node_modules/regenerate-unicode-properties Completed in 55637ms
npm timing reifyNode:node_modules/ajv-formats/node_modules/ajv Completed in 56497ms
npm timing reifyNode:node_modules/@angular-devkit/schematics/node_modules/ajv Completed in 56472ms
[##################] \ reify:ajv: timing reifyNode:node_modules/@angular-devkit/schematics/node_modules/ajv Completed in 564

The only thing that I can think of right now is that Bit Defender (the only other application running) is interfering somehow, however it's the one application I can't turn off.

I've seen this issue occur on different projects, on different network and internet connections, and on different machines. Does anyone have any advice on how to investigate this, or at the very least a way to kill the process when it hangs like this without having to reboot the machine? Being forced to reboot when this issue occurs is perhaps the most frustrating thing in all of this.

Expected Behavior

npm install should either succeed or show an error. If it gets stuck it should either time-out or be closable by the user.

Steps To Reproduce

Clear down the node_modules folder (ie with something like rmdir /q /s)
Run. npm install
Watch and wait.
If it succeeds, repeat the above steps until the freeze is observed.

Environment

npm: 8.1.3
Node: v16.13.0
OS: Windows 10 Version 21H1 (OS Build 19043.1288)
platform: Lenovo ThinkPad

npm config:


; "builtin" config from C:\Users\<REDACTED>\AppData\Roaming\npm\node_modules\npm\npmrc

prefix = "C:\Users\\AppData\Roaming\npm"

; "user" config from C:\Users\.npmrc

//pkgs.dev.azure.com//_packaging//npm/registry/:_authToken = (protected)

; node bin location = C:\Program Files\nodejs\node.exe ; cwd = C:\Users\ ; HOME = C:\Users\ ; Run npm config ls -l to show all defaults.

jmashore commented 5 days ago

I reverted back to npm@10.3.0 and this resolved the issue. It may have been just happenstance but for now, it appears to work properly.

Previous npm info using npm@10.9.0 npm info using node@v22.8.

After : Ubuntu Jammy npm info using npm@10.3.0 npm info using node@v22.8.0

npm install -g npm@10.3.0

Tofandel commented 3 days ago

I have found the culprit

https://github.com/npm/cli/blob/780afc50e3a345feb1871a28e33fa48235bc3bd5/workspaces/arborist/lib/arborist/build-ideal-tree.js#L193-L213 It's coming from checkPlatform which is done in a loop over each node, inside checkPlatform there is this line https://github.com/npm/npm-install-checks/blob/4751cb40207cf48ac21ac041b0c239e3c10d37e0/lib/current-env.js#L24

This line takes 40seconds by itself to execute (node bug) and the fact is that it's also not promisified in the loop so this is done sequentially for all the 2000 packages in the tree, this effectively would take in theory a whole 70 days for my upgrade command to complete (that's a lot of time to wait 😉)

ljharb commented 3 days ago

It's done concurrently, not sequentially, because it's not awaited in the loop.

Tofandel commented 3 days ago

But it's not async https://github.com/npm/npm-install-checks/blob/main/lib/index.js#L25 so yes it's done sequentially, not concurrently at all, the async keyword in front of #checkEngineAndPlatform does absolutely nothing

ljharb commented 3 days ago

ah, yes, good point :-)

npm / cli