search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.5k stars 3.19k forks source link

[BUG] npm install hangs for ~2 minutes with optional dependencies in npm 10.9.0 #7900

Open Guria opened 1 week ago

Guria commented 1 week ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

When using npm 10.9.0, npm install hangs for approximately 2 minutes during the idealTree phase when package.json includes optional dependencies that return 404. The same package.json installs in ~5s with npm 10.8.2.

npm hangs during the idealTree phase for ~2 minutes Eventually completes successfully Time measurement: npm install takes 2+ minutes

Detailed logs and measurements can be found in repro repository

Expected Behavior

Quick failure/skip of optional dependencies Similar to npm 10.8.2 behavior which completes in ~5s

Steps To Reproduce

Repository with minimal reproduction: https://github.com/Guria/npm10.9.0-hang-repro

Create package.json:

{
  "name": "npm-hang-repro",
  "version": "1.0.0",
  "description": "Reproduction for npm install hanging issue",
  "optionalDependencies": {
    "@nonexistent/package1": "^1.0.0",
    "@nonexistent/package2": "^1.0.0"
  },
  "dependencies": {
    "express": "^4.18.0"
  }
}

Steps to reproduce:

# Install npm 10.9.0
npm i -g npm@10.9.0

# Clean environment
npm cache clean --force
rm -rf node_modules package-lock.json

# Install with timing
time npm install --verbose --timing --loglevel silly

Environment

OS: Linux (Manjaro 6.10.13-3) npm: 10.9.0 unaffected npm: 10.8.2 Node.js: both behaviors confirmed on v20.18.0 and v22.11.0

; node bin location = /home/aleksei_gurianov/.asdf/installs/nodejs/20.18.0/bin/node
; node version = v20.18.0
; npm local prefix = /home/aleksei_gurianov/ws
; npm version = 10.9.0
; cwd = /home/aleksei_gurianov/ws
; HOME = /home/aleksei_gurianov
; Run `npm config ls -l` to show all defaults.

Additional versions checks

Issue reproduces with npm@10.9.0 on all recent node versions:

No issue with npm@10.8.2 on all above versions.

Guria commented 1 week ago

Just to clarify why using non-existing packages in optionalDependencies is a valid real-world scenario:

This pattern is commonly used when packages are optionally required based on environment. Some examples:

  1. Enterprise/Community edition split:

    // Only load enterprise features if the package is available
try {
 enterpriseFeatures = require('@company/enterprise-features');
} catch (e) {
 // Fall back to community features
}

  2. Platform-specific dependencies:

    // Try to load platform-specific optimizations
try {
 nativeBindings = require('@org/native-optimizations');
} catch (e) {
 // Use JS fallback
}

  3. Internal packages that are only available within corporate network/registry:

    // Load internal tooling if available
try {
 internalTools = require('@internal/build-tools');
} catch (e) {
 // Use public alternatives
}

In all these cases, npm should gracefully skip unavailable optional dependencies rather than hanging. The 2-minute delay makes the development experience significantly worse, especially in CI/CD environments where multiple clean installs might be required.

wickedest commented 1 week ago

I think I am hitting this issue too. I'm running an install using gitlab-ci-local. It's using npm 10.9.0 (node:22-bookworm). For me, it never finishes (I killed it after 5m). For me, it hangs in idealTree. I ran it using npm 10.8.3 and placed both outputs side-by-side. image

Tofandel commented 1 week ago

This is a duplicate of https://github.com/npm/cli/issues/4028

Guria commented 1 week ago

I am not sure it is exactly same. In my case I have stable reproduction starting with 10.9.0, and no issue with 10.8.2.

Linked issue has been reported on earlier versions.

Looks more like #7814 and #7868.

Tofandel commented 1 week ago

Which are also the same issue and on both it is either reported as starting in either 10.4.0 or 10.9.0, it is all an issue in the checkPlatform method, it has 2 different code paths which use it and maybe different resolution behaviors happen with different npm versions but at the core it's the same issue

Try to run your command with --libc="glibc" and it should fix it if you only hit the issue past 10.8.2, if not use --force instead

paulrutter commented 1 week ago

I encountered a similar issue, while in my case npm install would never finish. I narrowed it down to the isolated-vm dependency, which would never finish compiling (hanging on a gcc command randomly, viewed via --foreground-scripts).

The workaround that worked for me is setting --jobs 10 so compilation of native code doesn't take all CPU cores. See https://github.com/nodejs/node-gyp?tab=readme-ov-file#command-options.

npm install --jobs 10 // or npm ci --jobs 10

It's not a pretty solution, and maybe it's not the same issue as described here, but it also hung upon npm install for an optional dependency in my case.

Node version: 22.11.0 NPM version: 10.9.0 (downgrading to 10.3.0 as suggested in this thread didn't help for me though) Platform: Linux, Dockerized, Amazon Linux 2023 base image

VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.6.20241010"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2028-03-15"

gcc --version
gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Hope this is helpful for others.

kchindam-infy commented 1 week ago

Based on testing on windows with node 23 and npm 10.9.0, I did not encounter the npm install hanging issue. Try using the node 23 to see if it resolves the issue.

paulrutter commented 1 week ago

Based on testing on windows with node 23 and npm 10.9.0, I did not encounter the npm install hanging issue. Try using the node 23 to see if it resolves the issue.

I'm keen on using LTS versions only, so even if that solves the issue, it's not a good solution until it would land in node 22 as well.

Guria commented 1 week ago

@paulrutter It wasn't suggestion to use non LTS version, but just to test if your case is still reproduced on this version in order to help triage

paulrutter commented 1 week ago

@paulrutter It wasn't suggestion to use non LTS version, but just to test if your case is still reproduced on this version in order to help triage

Sorry, i misunderstood. I can try that next week.

Tofandel commented 1 week ago

@paulrutter your issue is the only one that seems different from this issue and all the other linked issue, especially given the fact you still had it on 10.3.0

paulrutter commented 1 week ago

@paulrutter your issue is the only one that seems different from this issue and all the other linked issue, especially given the fact you still had it on 10.3.0

Yes, true. I suspect it's an issue with that specific dependency, hence i also created an issue there.

Tofandel commented 9 hours ago

@Guria This should be fixed in npm 10.9.1, could you try that and confirm?