npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.46k stars 3.15k forks source link

[BUG] npx does not fetch latest possible semvar match #7838

Open jeff-an opened 2 weeks ago

jeff-an commented 2 weeks ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

When using the syntax npx <package>@<semvar> <command>, npx is always using a local cached version instead of fetching the latest available version that falls within the semvar from the npm registry and prompting for an upgrade.

Running npm cache clean --force does not seem to help.

The issue only seems to be reproducible on some machines. One user even reported that with momentic@1.0.12 installed locally, npx momentic^1 was still invoking 1.0.11 instead of the newer version.

Expected Behavior

I expect npx to issue a prompt like the one below:

Need to install the following packages:
momentic@1.0.13
Ok to proceed? (y)

rather than proceeding with the locally cached version of momentic@1.0.12, for example.

Steps To Reproduce

  1. Run npx momentic@1.0.12 init and accept the install prompt. Ignore the output of the program (the program in this case doesn't matter and can be substituted with any other).
  2. Run npx momentic@^1 init. This should be expected to prompt to install 1.0.13 or whatever the latest version is. However, it does not and instead prints the same output as step 1.

Screenshot of what I mean on the turbo repo (the latest turbo version is 2.1.3 at time of writing): Screenshot 2024-10-15 at 3 56 24 PM

Environment

auto-install-peers = true public-hoist-pattern = ["eslint-plugin","prisma","bull"]


I confirmed that my npx path is fixed and set to:

which npx /Users//.nvm/versions/node/v20.9.0/bin/npx

milaninfy commented 2 weeks ago

I am getting expected behaviour

~/workarea/rep/test $ npx -ddd momentic@1.0.12 init
Need to install the following packages:
momentic@1.0.12
Ok to proceed? (y) y
~/workarea/rep/test $ npx momentic@^1 init
Need to install the following packages:
momentic@1.0.13
Ok to proceed? (y) 
jeff-an commented 2 weeks ago

Thanks for the responses folks! --no-cache and prefer-online both do not seem to help this case: Screenshot 2024-10-16 at 4 18 39 PM

We know that it works on some people's machines but not others. How can we debug why? At this point we are thinking of just hitting npm's registry programmatically at startup to figure out what the latest version is.

ljharb commented 2 weeks ago

you don't need to do that; do npx foo@latest and you'll get the latest no matter what's locally available.

jeff-an commented 2 weeks ago

We are aware of that, but we don't want to use @latest because it will automatically install versions that may be backwards incompatible with what the user is currently using.

Besides, it seems like a bug that this behavior is a) non-deterministic across machines and b) different from what is advertised in the official docs:

Package names with a specifier will only be considered a match if they have the exact same name and version as the local dependency.
milaninfy commented 2 weeks ago

@jeff-an what's the output of npm -v and npm config ls -a

jeff-an commented 2 weeks ago

I put it in the environment section:

version: 10.9.0

npm config:


auto-install-peers = true
public-hoist-pattern = ["*eslint-plugin*","*prisma*","*bull*"]
milaninfy commented 1 week ago

npx will first check in local project/workspaces from where you are running the command to see if matching range version is found, if not then check globally and then pull from registry. So if you are running npx command in a folder where this package is already installed or part of node_modules then it would use that if it's matching.

jeff-an commented 1 week ago

What constitutes a local project or workspace? We have not installed this package (momentic) anywhere - it is only invoked as a CLI. It never appears as an entry in any package.json in our working tree or above.

milaninfy commented 1 day ago

Project with package.json and dependencies installed or this cli tool installed globally. unless it's installed locally on project from where you are running the command or globally installed. it should get the correct version based on range or version specified. However at my end it's not reproducible even with node 20.9.0 and npx 10.9.0. It does fetch correct values Please provide verbose logs of these runs if possible.

My output


~/workarea/rep $ node -v                               
v20.9.0
~/workarea/rep $ npm -v
10.9.0
~/workarea/rep $ npx -v
10.9.0
~/workarea/rep $ npx turbo@2.1.0 -V                    
Need to install the following packages:
turbo@2.1.0
Ok to proceed? (y) 

 ERROR  unexpected argument '-V' found

  tip: to pass '-V' as a value, use '-- -V'

Usage: turbo [OPTIONS] [COMMAND]

For more information, try '--help'.

~/workarea/rep $ npx turbo@^2 -V                       
Need to install the following packages:
turbo@2.2.3
Ok to proceed? (y) 

 ERROR  unexpected argument '-V' found

  tip: to pass '-V' as a value, use '-- -V'

Usage: turbo [OPTIONS] [COMMAND]

For more information, try '--help'.

~/workarea/rep $ npm config ls
; "project" config from /Users/milaninfy/workarea/rep/.npmrc

auto-install-peers = true
public-hoist-pattern = "[\"*eslint-plugin*\",\"*prisma*\",\"*bull*\"]"

~/workarea/rep $ 
jeff-an commented 1 day ago

What kind of debug logs can we provide? Unfortunately it does not appear npx has a --debug or --verbose mode that prints more information about how its resolving. A colleague of ours running on windows just encountered the problem again yesterday. Here's the information from his machine: Screenshot 2024-10-30 at 11 40 56 AM

We confirmed that there is no package.json in the current directory where he was running the command. Will try to get npm list -g information as well.

jeff-an commented 1 day ago

Screenshot 2024-10-30 at 2 57 11 PM

npm list -g showing nothing installed globally

milaninfy commented 16 hours ago

you can use command this way npx -ddd turbo@^2 -V to enable silly logs

jeff-an commented 10 hours ago

Screenshot 2024-10-31 at 11 53 06 AM

Screenshot 2024-10-31 at 11 57 12 AM

Here's the output from my laptop and a repro of the bug

jeff-an commented 10 hours ago

also repros in my tmp folder, where there is no package.json:

Screenshot 2024-10-31 at 11 57 40 AM