Custom node-fetch causes builds to fail

[kuzdogan]

Recently we removed the ipfs dependency in our repository to opt-in for the lightweight ipfs-core package for our tests. Our repository already has node-fetch package and following the installation of ipfs-core@0.12.2 and subsequently ipfs-utils@9.0.2 our builds started to fail.

The reason seems to be the custom node package used in this repo: node-fetch@npm:@achingbrain/node-fetch@2.6.7. Our other node-fetch packages had @^2.6.x and since the latest official package is 2.6.6 the custom @acingbrain/node-fetch@2.6.7 overrides others.

uzdogan@kuzdogan-UX331UN:~/repos/sourcify$ npm ls node-fetch
ethereum-sourcify@1.0.0 /home/kuzdogan/repos/sourcify
├─┬ @ethereum-sourcify/core@0.2.10
│ └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
├─┬ @ethereum-sourcify/verification@0.2.10
│ └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
├─┬ ipfs-core@0.12.2
│ ├─┬ ipfs-unixfs-importer@9.0.6
│ │ └─┬ rabin-wasm@0.1.5
│ │   └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
│ └─┬ ipfs-utils@9.0.2
│   ├─┬ native-fetch@3.0.0
│   │ └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
│   └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
├─┬ lerna@3.22.1
│ └─┬ @lerna/version@3.22.1
│   ├─┬ @lerna/github-client@3.22.0
│   │ └─┬ @octokit/rest@16.43.2
│   │   └─┬ @octokit/request@5.6.2
│   │     └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
│   └─┬ @lerna/gitlab-client@3.15.0
│     └── node-fetch@npm:@achingbrain/node-fetch@2.6.7 deduped
└── node-fetch@npm:@achingbrain/node-fetch@2.6.7

We get the following error in our builds.

npx lerna bootstrap
npx: installed 673 in 35.178s
lerna notice cli v4.0.0
lerna info ci enabled
lerna info Bootstrapping 4 packages
lerna info Installing external dependencies
lerna ERR! npm ci --no-package-lock exited 1 in 'ethereum-sourcify'
lerna ERR! npm ci --no-package-lock stderr:
npm ERR! Invalid Version: npm:@achingbrain/node-fetch@2.6.7

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/circleci/.npm/_logs/2021-11-29T13_58_42_504Z-debug.log
lerna ERR! npm ci --no-package-lock exited 1 in 'ethereum-sourcify'
lerna WARN complete Waiting for 3 child processes to exit. CTRL-C to exit immediately

Reproducing

To reproduce follow these steps.

Our builds are on CircleCI. Spin up the CircleCI node container

$ docker run -it circleci/node:14

In the container terminal cd into circleci folder

$ cd home/circleci

Clone the repository and cd

$ git clone https://github.com/ethereum/sourcify.git && cd sourcify

Checkout the relevant commit. (See the branch on GitHub )

$ git checkout a8c696082165ad7230d9a825563b165d18dc15d9

Install dependencies

$ npx lerna bootstrap --ci

Note that --ci flag is used. This is automatically detected in the CircleCI build environment. Local installs without this flag will not fail.

Fix

I was able to fix the issue by setting the node-fetch@^2.6.1 on our repo to fixed node-fetch@2.6.6. Then removing the package-lock.json files and running an npm install.

This results with two different node-fetch modules being used as intended:

$ npm ls node-fetch
ethereum-sourcify@1.0.0 /home/kuzdogan/repos/sourcify
├─┬ @ethereum-sourcify/core@0.2.10
│ └── node-fetch@2.6.6 deduped
├─┬ @ethereum-sourcify/verification@0.2.10
│ └── node-fetch@2.6.6 deduped
├─┬ ipfs-core@0.12.2
│ ├─┬ ipfs-unixfs-importer@9.0.6
│ │ └─┬ rabin-wasm@0.1.5
│ │   └── node-fetch@2.6.6 deduped
│ └─┬ ipfs-utils@9.0.2
│   ├─┬ native-fetch@3.0.0
│   │ └── node-fetch@2.6.6 deduped
│   └── node-fetch@npm:@achingbrain/node-fetch@2.6.7
├─┬ lerna@3.22.1
│ └─┬ @lerna/version@3.22.1
│   ├─┬ @lerna/github-client@3.22.0
│   │ └─┬ @octokit/rest@16.43.2
│   │   └─┬ @octokit/request@5.6.2
│   │     └── node-fetch@2.6.6 deduped
│   └─┬ @lerna/gitlab-client@3.15.0
│     └── node-fetch@2.6.6 deduped
└── node-fetch@2.6.6

A better way to handle this would be to name the custom node-fetch in this repo with an alias so that it does not interfere with other node-fetch versions.

[welcome[bot]]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[lidel]

Sorting out node-fetch situation is wip, see #154

[ShynRou]

Any updates?

[achingbrain]

Fixed by #244