search

NomicFoundation / edr

An Ethereum development runtime implementation that can be reused to build new developer tools.
MIT License
57 stars 12 forks source link

Performance regression when forking with hardhat EDR #343

Closed fullkomnun closed 6 months ago

fullkomnun commented 7 months ago

Version of Hardhat

2.22.1

What happened?

While unit test and deployment scripts (using 'hardhat-deploy' plugin) that run against the hardhat network with no forking appear to execute faster after updating to v2.22.1, when running tests or deployment scripts against the hardhat network with forking any remote network (with or without setting a pinned block number, public networks through Alchemy or remote internal Geth node) I see execution time is increased by about 50% compared to pre-EDR hardhat releases. Verified that even tests not using hardhat-deploy suffer from longer runtime when using forking. Let me know what can I do to provide additional useful details if you unaware of this issue.

Minimal reproduction steps

Any test when forking is running significantly slower comapred to pre-EDR releases. Node: v20.10 OS: macOS CPU: Apple Silicon (arm64)

Can't yet verify execution times on Github ubuntu runner since I'm investigating a crash in EDR rust code when running tests

Search terms

EDR

agostbiro commented 7 months ago

Hi @fullkomnun thanks for reporting this issue!

I tried a quick repro with a sample project here and I couldn't reproduce the issue unfortunately. With hardhat=2.22.2 (latest EDR release) I'm getting ~800ms median execution time for npx hardhat test with a warm cache. With hardhat=2.20.1 (latest non-EDR release) I'm getting ~1000ms median execution time with a warm cache, so EDR is ~20% faster.

Two things changed with the EDR release that could cause the problem that you're seeing:

  1. The RPC cache path is slightly different. In the linked sample project the EDR RPC cache is cache/hardhat-network-fork/rpc_cache (relative to the repo root). Could you verify if this cache is reused between your test executions?
  2. EDR is being more conservative about what is a safe block number to fork from. So even if you pinned the block number and it's very close to the latest, it might not get cached. For Ethereum mainnet we're using a block depth of 32 as the safe block number and default to 128 for unknown networks. Could you retest with a pinned block number that is at least 1000 behind the latest block?

If none of these resolve the issue, could you please share more repro steps?

fvictorio commented 6 months ago

Closing for lack of response.

fullkomnun commented 4 months ago

Sorry for lack of response, the reason for longer deployment tests were related to re-deployment of contracts (and therefore longer pre-tests setup) due to targetting 'shangahi' hardfork instead of 'merge'/'paris' which due to the introduction of push0 opcode affect almost every contract and triggers re-deployment.

EDR is fine =]

fvictorio commented 4 months ago

Thanks for letting us know!