search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.34k stars 3.07k forks source link

[BUG] devDependencies of project with npm-shrinkwrap.json are getting installed #4323

Open RandomByte opened 2 years ago

RandomByte commented 2 years ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

When installing a package (npm install <package name>) from the npm registry that contains an npm-shrinkwrap.json, dependencies listed with "dev": true are installed unexpectedly.

Expected Behavior

devDependencies of the published package should not be installed when consuming the package.

Steps To Reproduce

Given: Node.js v16.13.2 and npm v8.3.2

I created an example project to showcase this issue: https://github.com/RandomByte/npm-shrinkwrap-test

  1. git clone git@github.com:RandomByte/npm-shrinkwrap-test.git && cd npm-shrinkwrap-test
  2. npm install
  3. Check for known devDependency "ava" and notice that it has been installed: 
    npm ls -a | grep 'ava'
> ├─┬ ava@3.15.0 extraneous
  4. npm prune --production removes the ava package. But a following npm install will install it again

It seems that after installing @ui5/cli@next in the empty project, its devDependencies where installed as normal dependencies. This is unexpected since a consumer of the package does not need its devDependencies.

In the past (with npm v6) we worked around this issue by executing npm prune --production followed by npm shrinkwrap during publishing of the @ui5/cli package. This basically removed all devDependencies from the npm-shrinkwrap.json (see https://github.com/SAP/ui5-tooling/issues/283#issuecomment-599985109). But this does not seem to work with npm v8 anymore, as the npm-shrinkwrap.json is not being updated after the prune took place.

I think the fundamental issue is present since at least npm v6. Relevant issues are:

Environment

bkendall commented 2 years ago

I'm going to +1 this via comment as well for visibility: I just worked around this in part of our build process for firebase-tools; it caused our prebuilt binaries to bloat from 130MB to 800MB+ and had to work pretty hard to find a workaround (thankfully we found one).

nlf commented 2 years ago

confirmed bug, we're ignoring the dev flag in the npm-shrinkwrap.json, installing the dev dependencies and flagging them as extraneous when we write the package-lock.json.

i confirmed the npm-shrinkwrap.json flags the relevant deps as dev.

as noted, npm prune --production removes the extraneous deps from node_modules but not from the package-lock, so a follow up npm install puts them right back.

as a further note, npm install --only=prod does skip reifying the dev deps, as does npm install --omit=dev so we are internally identifying them correctly, just for some reason when we reify a dep with a shrinkwrap we're not applying the default rules for dev dependencies.

bkendall commented 2 years ago

@nlf any chance that this can get prioritized?

Another interesting issue that comes with this is that npm audit (which probably shouldn't be running when a module is installed globally in our case, but that's another bug) checks all the dev dependencies that were installed too, which creates more noise for end users of our shrinkwrap'd CLI.

haines commented 2 years ago

It's also quite frustrating with Dependabot updates, because Dependabot uses npm install --package-lock-only which does not result in the shrinkwrapped devDependencies being installed. This creates churn with Dependabot removing the extraneous package-lock.json entries and then local npm install reintroducing them.

bkendall commented 2 years ago

This is something that we're scratching our head to solve. Installing all our dev dependencies when installing firebase-tools globally takes up a ton of space, and it'd be nice to not do that. We've come across other options like clean-publish but working around this issue in npm seems like not the correct way to do it long term.

It'd be really appreciated if this could be addressed soon! Thanks!

bkendall commented 2 years ago

Another week, another person surprised when dev dependencies are installed and it triggers other audit warnings too. It would be really great to get this fixed.

https://github.com/firebase/firebase-tools/issues/4586

bkendall commented 2 years ago

Once again, another person has escalated an issue to me where the dev dependencies are being installed. Would love to see this addressed soon.

cupidchan commented 2 years ago

Is there any timeline for this issue? It's pretty confusing to see so many irrelevant errors/warnings.

antonstefer commented 2 years ago

Commenting for visibility. Not sure if you're responsible @nlf but this deserves some attention.

cupidchan commented 2 years ago

This issue has been opened for a while now and we still see those irrelevant warnings and errors. Can this be prioritized and fixed in the near future? Thanks!

bkendall commented 1 year ago

I've given up on this ever being fixed and worked around it using clean-publish. I'd like to get back to using npm directly, but cutting the installed size by 75% just cannot be a benefit that I can wait on any longer.

RandomByte commented 6 months ago

Update: This can still be reproduced as described with Node v20.11.1 and npm v10.2.4