downloading the same compiler multiple times

drortirosh commented 1 year ago

Version of Hardhat

2.12.4

What happened?

We want to use the "viaIR" mode for several files, but it is too slow to compile the entire tests around it. So we use overrides in the compiler settings here

However, this causes hardhat to report (and probably download) the compiler multiple times. e.g. I changed the version to 0.8.21 and got:

> hardhat compile

Solidity 0.8.21, 0.8.21, 0.8.21 are not fully supported yet. You can still use Hardhat, but some features, like stack traces, might not work correctly.

Learn more at https://hardhat.org/hardhat-runner/docs/reference/solidity-support

Downloading compiler 0.8.21
Downloading compiler 0.8.21
Downloading compiler 0.8.21

Minimal reproduction steps

checkout git@github.com:eth-infinitism/account-abstraction.git
change compiler version in hardhat.config.ts
yarn compile

Search terms

No response

schaable commented 1 year ago

I can confirm this is happening. Thanks @drortirosh! We'll take a look as soon as we can.

michaelbnewman commented 3 months ago

Here's some testing/analysis that may help decide a path forward. I skipped changing the compiler version to see what would happen:

Base Case (5 downloading messages)

$ docker run -it --rm node:20 bash
$ git clone https://github.com/eth-infinitism/account-abstraction.git
$ cd account-abstraction
$ yarn install
$ yarn compile
yarn run v1.22.22
$ ./scripts/hh-wrapper compile
Downloading compiler 0.8.23
Downloading compiler 0.8.23
Downloading compiler 0.8.23
Downloading compiler 0.8.23
Downloading compiler 0.8.23
Generating typings for: 95 artifacts in dir: typechain for target: ethers-v5
Successfully generated 260 typings!
Compiled 92 Solidity files successfully (evm target: paris).
Done in 38.11s.

Add some debug messaging to track what is downloaded

$ docker run -it --rm node:20 bash
$ git clone https://github.com/eth-infinitism/account-abstraction.git
$ cd account-abstraction
$ yarn install

$ sed -i '/const { getGlobalDispatcher, ProxyAgent, request } = await Promise.resolve().then(() => __importStar(require("undici")));/a\
        console.log(`filePath: ${filePath}`);\
' node_modules/hardhat/internal/util/download.js

$ yarn compile
yarn run v1.22.22
$ ./scripts/hh-wrapper compile
Downloading compiler 0.8.23
filePath: /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/list.json
Downloading compiler 0.8.23
Downloading compiler 0.8.23
filePath: /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/solc-linux-amd64-v0.8.23+commit.f704f362
filePath: /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/solc-linux-amd64-v0.8.23+commit.f704f362
Downloading compiler 0.8.23
filePath: /root/.cache/hardhat-nodejs/compilers-v2/wasm/list.json
filePath: /root/.cache/hardhat-nodejs/compilers-v2/wasm/soljson-v0.8.23+commit.f704f362.js
filePath: /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/solc-linux-amd64-v0.8.23+commit.f704f362
Downloading compiler 0.8.23
Downloading compiler 0.8.23
filePath: /root/.cache/hardhat-nodejs/compilers-v2/wasm/soljson-v0.8.23+commit.f704f362.js
filePath: /root/.cache/hardhat-nodejs/compilers-v2/wasm/soljson-v0.8.23+commit.f704f362.js
Generating typings for: 95 artifacts in dir: typechain for target: ethers-v5
Successfully generated 260 typings!
Compiled 92 Solidity files successfully (evm target: paris).
Done in 25.81s.

Attempt to avoid multiple downloads


$ docker run -it --rm node:20 bash
$ git clone https://github.com/eth-infinitism/account-abstraction.git
$ cd account-abstraction
$ yarn install

$ sed -i '1i const fs = require("fs-extra");' node_modules/hardhat/internal/util/download.js

$ sed -i '/const { getGlobalDispatcher, ProxyAgent, request } = await Promise.resolve().then(() => __importStar(require("undici")));/a\
    if (await fs.pathExists(filePath)) {\
        console.log(`File already exists at ${filePath}, skipping download.`);\
        return;\
    }' node_modules/hardhat/internal/util/download.js

$ yarn compile
yarn run v1.22.22
$ ./scripts/hh-wrapper compile
Downloading compiler 0.8.23
Downloading compiler 0.8.23
Downloading compiler 0.8.23
File already exists at /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/solc-linux-amd64-v0.8.23+commit.f704f362, skipping download.
Downloading compiler 0.8.23
File already exists at /root/.cache/hardhat-nodejs/compilers-v2/linux-amd64/solc-linux-amd64-v0.8.23+commit.f704f362, skipping download.
Downloading compiler 0.8.23
Downloading compiler 0.8.23
File already exists at /root/.cache/hardhat-nodejs/compilers-v2/wasm/soljson-v0.8.23+commit.f704f362.js, skipping download.
File already exists at /root/.cache/hardhat-nodejs/compilers-v2/wasm/soljson-v0.8.23+commit.f704f362.js, skipping download.
Generating typings for: 95 artifacts in dir: typechain for target: ethers-v5
Successfully generated 260 typings!
Compiled 92 Solidity files successfully (evm target: paris).
Done in 24.96s.

So there are separate linux-amd64 and wasm versions being downloaded. The final example will skip extra downloads, but doesn't change the code enough to avoid the "Downloading compiler" messages.

NomicFoundation / hardhat