search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.36k stars 3.09k forks source link

[BUG] Broken package-lock/node_modules with missing/extraneous dependency #3267

Open mshima opened 3 years ago

mshima commented 3 years ago

Is there an existing issue for this?

Current Behavior

A freshly installed package.json contains missing and extraneous dependencies.

npm ERR! code ELSPROBLEMS
npm ERR! extraneous: @typescript-eslint/types@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/types
npm ERR! extraneous: @typescript-eslint/typescript-estree@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/typescript-estree
npm ERR! missing: jest-preset-angular@^8.4.0, required by @angular-builders/jest@11.1.1
npm ERR! missing: @typescript-eslint/experimental-utils@4.16.1, required by @angular-eslint/eslint-plugin@4.3.0
npm ERR! missing: @typescript-eslint/experimental-utils@4.24.0, required by @typescript-eslint/eslint-plugin@4.24.0
npm ERR! extraneous: @typescript-eslint/visitor-keys@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/typescript-estree/node_modules/@typescript-eslint/visitor-keys

A second npm install removes the extraneous dependencies, but don't add missing dependencies.

Expected Behavior

No response

Steps To Reproduce

  1. With this package.json:

    {
"dependencies": {
"@angular/common": "11.2.13",
"@angular/compiler": "11.2.13",
"@angular/core": "11.2.13",
"@angular/forms": "11.2.13",
"@angular/localize": "11.2.13",
"@angular/platform-browser": "11.2.13",
"@angular/platform-browser-dynamic": "11.2.13",
"@angular/router": "11.2.13",
"@fortawesome/angular-fontawesome": "0.8.2",
"@fortawesome/fontawesome-svg-core": "1.2.35",
"@fortawesome/free-solid-svg-icons": "5.15.3",
"@ng-bootstrap/ng-bootstrap": "9.1.1",
"@ngx-translate/core": "13.0.0",
"@ngx-translate/http-loader": "6.0.0",
"bootstrap": "4.6.0",
"dayjs": "1.10.4",
"ngx-infinite-scroll": "10.0.1",
"ngx-webstorage": "7.0.1",
"rxjs": "6.6.7",
"swagger-ui-dist": "3.48.0",
"tslib": "2.2.0",
"zone.js": "0.11.4"
},
"devDependencies": {
"@angular-builders/custom-webpack": "11.1.1",
"@angular-builders/jest": "11.1.1",
"@angular-eslint/eslint-plugin": "4.3.0",
"@angular/cli": "11.2.12",
"@angular/compiler-cli": "11.2.13",
"@angular/service-worker": "11.2.13",
"@types/jest": "26.0.23",
"@types/node": "15.3.0",
"@typescript-eslint/eslint-plugin": "4.24.0",
"@typescript-eslint/parser": "4.24.0",
"browser-sync": "2.26.14",
"browser-sync-webpack-plugin": "2.3.0",
"concurrently": "6.1.0",
"copy-webpack-plugin": "6.4.1",
"cypress": "6.6.0",
"cypress-audit": "0.3.0",
"eslint": "7.26.0",
"eslint-config-prettier": "8.3.0",
"eslint-plugin-cypress": "2.11.3",
"eslint-webpack-plugin": "2.5.4",
"friendly-errors-webpack-plugin": "1.7.0",
"husky": "4.3.8",
"jest": "26.6.3",
"jest-date-mock": "1.0.8",
"jest-junit": "12.0.0",
"jest-sonar-reporter": "2.0.0",
"lighthouse": "7.5.0",
"lint-staged": "11.0.0",
"merge-jsons-webpack-plugin": "1.0.21",
"prettier": "2.3.0",
"prettier-plugin-java": "1.0.2",
"prettier-plugin-packagejson": "2.2.11",
"rimraf": "3.0.2",
"simple-progress-webpack-plugin": "2.0.0",
"typescript": "4.1.5",
"wait-on": "5.3.0",
"webpack": "4.46.0",
"webpack-bundle-analyzer": "4.4.2",
"webpack-merge": "5.7.3",
"webpack-notifier": "1.13.0"
},
"engines": {
"node": ">=14.16.0"
}
}

  2. Run 'npm install'

  3. Run 'npm ls --depth 10'

    npm ERR! code ELSPROBLEMS
npm ERR! extraneous: @typescript-eslint/types@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/types
npm ERR! extraneous: @typescript-eslint/typescript-estree@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/typescript-estree
npm ERR! missing: jest-preset-angular@^8.4.0, required by @angular-builders/jest@11.1.1
npm ERR! missing: @typescript-eslint/experimental-utils@4.16.1, required by @angular-eslint/eslint-plugin@4.3.0
npm ERR! missing: @typescript-eslint/experimental-utils@4.24.0, required by @typescript-eslint/eslint-plugin@4.24.0
npm ERR! extraneous: @typescript-eslint/visitor-keys@4.16.1 /Users/mshima/aplicacoes/bug-tracker/node_modules/@typescript-eslint/typescript-estree/node_modules/@typescript-eslint/visitor-keys

  4. Run npm install

    removed 3 packages, and audited 2284 packages in 3s

  5. Run 'npm ls --depth 10'

    npm ERR! code ELSPROBLEMS
npm ERR! missing: jest-preset-angular@^8.4.0, required by @angular-builders/jest@11.1.1
npm ERR! missing: @typescript-eslint/experimental-utils@4.16.1, required by @angular-eslint/eslint-plugin@4.3.0
npm ERR! missing: @typescript-eslint/experimental-utils@4.24.0, required by @typescript-eslint/eslint-plugin@4.24.0

Environment

ljharb commented 3 years ago

Why are you running npm ls with a depth option at all? Due to hoisting, it’s kind of a useless argument - just run npm ls by itself.

mshima commented 3 years ago

Because npm ls doesn't show the problem:

$ npm ls
@
├── @angular-builders/custom-webpack@11.1.1
├── @angular-builders/jest@11.1.1
├── @angular-eslint/eslint-plugin@4.3.0
├── @angular/cli@11.2.12
├── @angular/common@11.2.13
├── @angular/compiler-cli@11.2.13
├── @angular/compiler@11.2.13
├── @angular/core@11.2.13
├── @angular/forms@11.2.13
├── @angular/localize@11.2.13
├── @angular/platform-browser-dynamic@11.2.13
├── @angular/platform-browser@11.2.13
├── @angular/router@11.2.13
├── @angular/service-worker@11.2.13
├── @fortawesome/angular-fontawesome@0.8.2
├── @fortawesome/fontawesome-svg-core@1.2.35
├── @fortawesome/free-solid-svg-icons@5.15.3
├── @ng-bootstrap/ng-bootstrap@9.1.1
├── @ngx-translate/core@13.0.0
├── @ngx-translate/http-loader@6.0.0
├── @types/jest@26.0.23
├── @types/node@15.3.0
├── @typescript-eslint/eslint-plugin@4.24.0
├── @typescript-eslint/parser@4.24.0
├── bootstrap@4.6.0
├── browser-sync-webpack-plugin@2.3.0
├── browser-sync@2.26.14
├── concurrently@6.1.0
├── copy-webpack-plugin@6.4.1
├── cypress-audit@0.3.0
├── cypress@6.6.0
├── dayjs@1.10.4
├── eslint-config-prettier@8.3.0
├── eslint-plugin-cypress@2.11.3
├── eslint-webpack-plugin@2.5.4
├── eslint@7.26.0
├── friendly-errors-webpack-plugin@1.7.0
├── husky@4.3.8
├── jest-date-mock@1.0.8
├── jest-junit@12.0.0
├── jest-sonar-reporter@2.0.0
├── jest@26.6.3
├── lighthouse@7.5.0
├── lint-staged@11.0.0
├── merge-jsons-webpack-plugin@1.0.21
├── ngx-infinite-scroll@10.0.1
├── ngx-webstorage@7.0.1
├── prettier-plugin-java@1.0.2
├── prettier-plugin-packagejson@2.2.11
├── prettier@2.3.0
├── rimraf@3.0.2
├── rxjs@6.6.7
├── simple-progress-webpack-plugin@2.0.0
├── swagger-ui-dist@3.48.0
├── tslib@2.2.0
├── typescript@4.1.5
├── wait-on@5.3.0
├── webpack-bundle-analyzer@4.4.2
├── webpack-merge@5.7.3
├── webpack-notifier@1.13.0
├── webpack@4.46.0
└── zone.js@0.11.4
ljharb commented 3 years ago

That means there is no problem. The validity of your dep graph is only assessable by including the entire dep graph.

mshima commented 3 years ago

So the problem affects npm ls too: See https://github.com/jhipster/generator-jhipster/runs/2625012481.

This is affecting every angular build since yesterday by this time. https://github.com/jhipster/generator-jhipster/actions/workflows/angular.yml (succeed build were skipped).

Triggering a rebuild at succeed builds also fails. Probably some dependency update broke the entire tree.

Error: Failed to load plugin '@typescript-eslint' declared in '.eslintrc.json': Cannot find module '@typescript-eslint/experimental-utils'
117
Require stack:
118
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/@typescript-eslint/eslint-plugin/dist/rules/adjacent-overload-signatures.js
119
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/@typescript-eslint/eslint-plugin/dist/rules/index.js
120
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/@typescript-eslint/eslint-plugin/dist/index.js
121
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/@eslint/eslintrc/lib/config-array-factory.js
122
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/@eslint/eslintrc/lib/index.js
123
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/eslint/lib/cli-engine/cli-engine.js
124
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/eslint/lib/eslint/eslint.js
125
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/eslint/lib/eslint/index.js
126
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/eslint/lib/cli.js
127
- /home/runner/work/generator-jhipster/generator-jhipster/app/node_modules/eslint/bin/eslint.js
128
Referenced from: /home/runner/work/generator-jhipster/generator-jhipster/app/.eslintrc.json
129
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
130
ljharb commented 3 years ago

ok - that issue (from eslint) is because @typescript-eslint/experimental-utils is missing, which it seems is perhaps the npm bug - that npm ls SHOULD be failing if it's actually missing?

mshima commented 3 years ago

@angular-builders/jest is missing too: npm ERR! missing: jest-preset-angular@^8.4.0, required by @angular-builders/jest@11.1.1.

$ npm ls --depth 10
...
├─┬ @angular-builders/jest@11.1.1
│ ├── @angular-devkit/architect@0.1102.13 deduped
│ ├── @angular-devkit/core@11.2.13 deduped
│ ├── UNMET DEPENDENCY jest-preset-angular@^8.4.0
mshima commented 3 years ago

Just a follow-up, adding those dependencies as direct dependencies fix the builds https://github.com/jhipster/generator-jhipster/pull/15059.

ruyadorno commented 2 years ago

hi @mshima is this still reproducible with the current version of the npm cli ? (npm@8.5.2 at the time of writing)