search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.44k stars 3.14k forks source link

[BUG] `npm i` fails for a package that exists in uplink, but, does not exist in private registry #5045

Open stagefright5 opened 2 years ago

stagefright5 commented 2 years ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

We have private registry. Our private registry has these 2 endpoints:

  1. GET /package/:artefactName/:version: This returns the metadata. If the package is not in the private repo, It queries the https://registry.npmjs.org/ for metadata and returns the response
  2. GET /package/:artefactName/-/:bundleName: This returns the tarball only if available in the private data store. Otherwise, throws error

We have added .npmrc with the following configs to one of our npm projects:

registry=http://localhost:8081/api/8.3.0/package
# package-lock=false
# save-prefix=

Issue:

See below logs for this issue where npm i fails.

Here is the verbose log of `npm i --logLevel verbose` ```log npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead. npm verb cli C:\Program Files\nodejs\node.exe C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js npm info using npm@8.12.2 npm info using node@v16.15.1 npm timing npm:load:whichnode Completed in 0ms npm timing config:load:defaults Completed in 1ms npm timing config:load:file:C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\npmrc Completed in 3ms npm timing config:load:builtin Completed in 3ms npm timing config:load:cli Completed in 2ms npm timing config:load:env Completed in 1ms npm timing config:load:file:D:\npm-i-test\.npmrc Completed in 1ms npm timing config:load:project Completed in 3ms npm timing config:load:file:C:\Users\Sankarshan\.npmrc Completed in 0ms npm timing config:load:user Completed in 1ms npm timing config:load:file:C:\Users\Sankarshan\AppData\Roaming\npm\etc\npmrc Completed in 0ms npm timing config:load:global Completed in 0ms npm timing config:load:validate Completed in 2ms npm timing config:load:credentials Completed in 1ms npm timing config:load:setEnvs Completed in 1ms npm timing config:load Completed in 15ms npm timing npm:load:configload Completed in 16ms npm timing npm:load:mkdirpcache Completed in 2ms npm timing npm:load:mkdirplogs Completed in 0ms npm verb title npm i npm verb argv "i" "--loglevel" "verbose" npm timing npm:load:setTitle Completed in 2ms npm timing config:load:flatten Completed in 3ms npm timing npm:load:display Completed in 14ms npm verb logfile logs-max:10 dir:C:\Users\Sankarshan\AppData\Local\npm-cache\_logs npm verb logfile C:\Users\Sankarshan\AppData\Local\npm-cache\_logs\2022-06-20T13_49_10_934Z-debug-0.log npm timing npm:load:logFile Completed in 19ms npm timing npm:load:timers Completed in 0ms npm timing npm:load:configScope Completed in 0ms npm timing npm:load Completed in 56ms npm timing arborist:ctor Completed in 0ms npm timing arborist:ctor Completed in 0ms npm timing idealTree:init Completed in 18ms npm timing idealTree:userRequests Completed in 0ms npm http fetch GET 200 http://localhost:8081/api/8.3.0/package/sprintf-js 408ms (cache miss) npm timing idealTree:#root Completed in 435ms npm timing idealTree:node_modules/sprintf-js Completed in 0ms npm timing idealTree:buildDeps Completed in 440ms npm timing idealTree:fixDepFlags Completed in 1ms npm timing idealTree Completed in 465ms npm timing reify:loadTrees Completed in 467ms npm timing reify:diffTrees Completed in 2ms npm timing reify:retireShallow Completed in 0ms npm timing reify:createSparse Completed in 6ms npm timing reify:loadBundles Completed in 0ms npm http fetch POST 404 http://localhost:8081/api/8.3.0/package/-/npm/v1/security/advisories/bulk 26ms npm http fetch POST 404 http://localhost:8081/api/8.3.0/package/-/npm/v1/security/audits/quick 13ms npm verb audit error HttpErrorGeneral: 404 Not Found - POST http://localhost:8081/api/8.3.0/package/-/npm/v1/security/audits/quick npm verb audit error at C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\lib\check-response.js:93:15 npm verb audit error at processTicksAndRejections (node:internal/process/task_queues:96:5) npm verb audit error at async Map.[getReport] (C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\arborist\lib\audit-report.js:333:21) npm verb audit error at async Map.run (C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\arborist\lib\audit-report.js:106:19) npm verb audit error HttpErrorGeneral: 404 Not Found - POST http://localhost:8081/api/8.3.0/package/-/npm/v1/security/audits/quick npm verb audit error at C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\lib\check-response.js:93:15 npm verb audit error at processTicksAndRejections (node:internal/process/task_queues:96:5) npm verb audit error at async Map.[getReport] (C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\arborist\lib\audit-report.js:333:21) npm verb audit error at async Map.run (C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\@npmcli\arborist\lib\audit-report.js:106:19) { npm verb audit error headers: [Object: null prototype] { npm verb audit error 'x-dns-prefetch-control': [ 'off' ], npm verb audit error 'x-frame-options': [ 'SAMEORIGIN' ], npm verb audit error 'strict-transport-security': [ 'max-age=15552000; includeSubDomains' ], npm verb audit error 'x-download-options': [ 'noopen' ], npm verb audit error 'x-content-type-options': [ 'nosniff' ], npm verb audit error 'x-xss-protection': [ '1; mode=block' ], npm verb audit error 'access-control-allow-origin': [ '*' ], npm verb audit error 'content-security-policy': [ "default-src 'self'" ], npm verb audit error 'content-type': [ 'text/html; charset=utf-8' ], npm verb audit error 'content-length': [ '188' ], npm verb audit error date: [ 'Mon, 20 Jun 2022 13:49:11 GMT' ], npm verb audit error connection: [ 'keep-alive' ], npm verb audit error 'keep-alive': [ 'timeout=5' ], npm verb audit error 'x-fetch-attempts': [ '1' ] npm verb audit error }, npm verb audit error statusCode: 404, npm verb audit error code: 'E404', npm verb audit error method: 'POST', npm verb audit error uri: 'http://localhost:8081/api/8.3.0/package/-/npm/v1/security/audits/quick', npm verb audit error body: , npm verb audit error pkgid: 'quick' npm verb audit error } npm timing auditReport:getReport Completed in 91ms npm timing reify:audit Completed in 92ms npm http fetch GET 500 http://localhost:8081/api/8.3.0/package/sprintf-js/-/sprintf-js-1.1.2.tgz 70274ms attempt #3 (cache skip) npm timing reify:rollback:createSparse Completed in 9ms npm timing reify:rollback:retireShallow Completed in 0ms npm timing command:i Completed in 70835ms npm verb stack HttpErrorGeneral: 500 Internal Server Error - GET http://localhost:8081/api/8.3.0/package/sprintf-js/-/sprintf-js-1.1.2.tgz npm verb stack at C:\Users\Sankarshan\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\lib\check-response.js:93:15 npm verb stack at processTicksAndRejections (node:internal/process/task_queues:96:5) npm verb statusCode 500 npm verb pkgid sprintf-js@http://localhost:8081/api/8.3.0/package/sprintf-js/-/sprintf-js-1.1.2.tgz npm verb cwd D:\npm-i-test npm verb Windows_NT 10.0.19044 npm verb node v16.15.1 npm verb npm v8.12.2 npm ERR! code E500 npm ERR! 500 Internal Server Error - GET http://localhost:8081/api/8.3.0/package/sprintf-js/-/sprintf-js-1.1.2.tgz npm verb exit 1 npm timing npm Completed in 71138ms npm verb unfinished npm timer reify 1655732951223 npm verb unfinished npm timer reify:unpack 1655732951730 npm verb unfinished npm timer reifyNode:node_modules/sprintf-js 1655732951731 npm verb code 1 npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Sankarshan\AppData\Local\npm-cache\_logs\2022-06-20T13_49_10_934Z-debug-0.log ```

Compare this to the yarn install --verbose log

Note: This installation succeeds if done in npm v6.x.x

Here is the verbose log of `yarn install --verbose` yarn version: 1.22.19 ```log verbose 0.2059042 Checking for configuration file "D:\\npm-i-test\\.npmrc". verbose 0.2067221 Found configuration file "D:\\npm-i-test\\.npmrc". verbose 0.2078709 Checking for configuration file "C:\\Users\\Sankarshan\\.npmrc". verbose 0.2084984 Found configuration file "C:\\Users\\Sankarshan\\.npmrc". verbose 0.2095982 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\npmrc". verbose 0.2103793 Checking for configuration file "D:\\npm-i-test\\.npmrc". verbose 0.2117668 Found configuration file "D:\\npm-i-test\\.npmrc". verbose 0.2157005 Checking for configuration file "D:\\npm-i-test\\.yarnrc". verbose 0.2165246 Found configuration file "D:\\npm-i-test\\.yarnrc". verbose 0.2176542 Checking for configuration file "C:\\Users\\Sankarshan\\.yarnrc". verbose 0.2184824 Found configuration file "C:\\Users\\Sankarshan\\.yarnrc". verbose 0.2194361 Checking for configuration file "C:\\Program Files\\nodejs\\etc\\yarnrc". verbose 0.2199986 Checking for configuration file "D:\\npm-i-test\\.yarnrc". verbose 0.2204214 Found configuration file "D:\\npm-i-test\\.yarnrc". verbose 0.2254752 current time: 2022-06-20T12:53:06.420Z info No lockfile found. [1/4] Resolving packages... verbose 0.2727569 Performing "GET" request to "http://localhost:8081/api/8.3.0/package/sprintf-js". verbose 0.4453627 Request "http://localhost:8081/api/8.3.0/package/sprintf-js" finished with status code 200. [2/4] Fetching packages... verbose 0.4788914 Performing "GET" request to "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.2.tgz". [3/4] Linking dependencies... verbose 0.7609122 Creating directory "D:\\npm-i-test\\node_modules\\sprintf-js". verbose 0.7661962 Creating directory "D:\\npm-i-test\\node_modules\\sprintf-js\\dist". verbose 0.7673571 Creating directory "D:\\npm-i-test\\node_modules\\sprintf-js\\src". verbose 0.7717975 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\CONTRIBUTORS.md" to "D:\\npm-i-test\\node_modules\\sprintf-js\\CONTRIBUTORS.md". verbose 0.7735682 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\CHANGELOG.md" to "D:\\npm-i-test\\node_modules\\sprintf-js\\CHANGELOG.md". verbose 0.774956 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\LICENSE" to "D:\\npm-i-test\\node_modules\\sprintf-js\\LICENSE". verbose 0.7759356 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\package.json" to "D:\\npm-i-test\\node_modules\\sprintf-js\\package.json". verbose 0.7769853 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\README.md" to "D:\\npm-i-test\\node_modules\\sprintf-js\\README.md". verbose 0.7780286 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\dist\\.gitattributes" to "D:\\npm-i-test\\node_modules\\sprintf-js\\dist\\.gitattributes". verbose 0.7789373 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\dist\\angular-sprintf.min.js" to "D:\\npm-i-test\\node_modules\\sprintf-js\\dist\\angular-sprintf.min.js". verbose 0.7799692 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\dist\\angular-sprintf.min.js.map" to "D:\\npm-i-test\\node_modules\\sprintf-js\\dist\\angular-sprintf.min.js.map". verbose 0.7810647 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\dist\\sprintf.min.js" to "D:\\npm-i-test\\node_modules\\sprintf-js\\dist\\sprintf.min.js". verbose 0.7823663 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\dist\\sprintf.min.js.map" to "D:\\npm-i-test\\node_modules\\sprintf-js\\dist\\sprintf.min.js.map". verbose 0.783775 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\src\\angular-sprintf.js" to "D:\\npm-i-test\\node_modules\\sprintf-js\\src\\angular-sprintf.js". verbose 0.7848578 Copying "C:\\Users\\Sankarshan\\AppData\\Local\\Yarn\\Cache\\v6\\npm-sprintf-js-1.1.2-da1765262bf8c0f571749f2ad6c26300207ae673-integrity\\node_modules\\sprintf-js\\src\\sprintf.js" to "D:\\npm-i-test\\node_modules\\sprintf-js\\src\\sprintf.js". [4/4] Building fresh packages... success Saved lockfile. Done in 0.64s. ```

Expected Behavior

After getting the metadata from the private registry, npm should resolve the package tarball from the url in the metadata. In this case, public npm registry

Steps To Reproduce

  1. Create a server that implements the below 2 end points:
    • GET /package/:artefactName/:version: This should return the metadata. If the package is not in the private repo, It should query the https://registry.npmjs.org/ for metadata and return the response
    • GET /package/:artefactName/-/:bundleName: This should return the tarball only if available in the private data store. Otherwise, throw error
  2. Create a folder npm-i-test
  3. cd to that folder and npm init -y
  4. create .npmrc with the following configs at the root of the project 
    registry=http://localhost:8081/api/8.3.0/package
# package-lock=false
# save-prefix=
  5. Clear the npm cache
  6. Install a publicly published package, for example, sprintf-js: npm i sprintf-js --logLevel verbose

Now, the npm i will get stuck for some time. After that it fails with the error similar to the above log

Environment

prefix = "C:\Users\Sankarshan\AppData\Roaming\npm"

; "user" config from C:\Users\Sankarshan.npmrc

//registry.npmjs.org/:_authToken = (protected)

; "project" config from D:\npm-i-test.npmrc

registry = "http://localhost:8081/api/8.3.0/package"

; node bin location = C:\Program Files\nodejs\node.exe ; node version = v16.15.1 ; npm local prefix = D:\npm-i-test ; npm version = 8.12.2 ; cwd = D:\npm-i-test ; HOME = C:\Users\Sankarshan ; Run npm config ls -l to show all defaults.

stagefright5 commented 2 years ago

Anyone looking into this or any updates?