search

ethereum / hevm

symbolic EVM evaluator
https://hevm.dev
GNU Affero General Public License v3.0
235 stars 48 forks source link

Increasing gas usage metric precision #603

Open rappie opened 6 days ago

rappie commented 6 days ago

I'm using Echidna's optimization mode to find the max amount of gas used by a specific function. However, have noticed significant discrepancies in HEVM's gas usage metrics compared to other tools.

The biggest discrepancy I've encountered in practice on a private codebase is 688957 (echidna) vs 716057 (medusa).

Benchmark: https://github.com/rappie/fuzzer-gas-metric-benchmark

d-xo commented 5 days ago

Interesting. Do you perhaps have any insights as to where the difference might be coming from?

I guess I had assumed that passing the general state tests would be sufficient to ensure accurate gas calculation, but it seems to not be the case :/

rappie commented 4 days ago

I'm afraid i'm not of much help here. I'm just a echidna (and medusa) user and not a dev.

You can see the code I'm using for the benchmark here: https://github.com/rappie/fuzzer-gas-metric-benchmark/blob/main/src/Benchmark.sol#L13-L17

If you can provide me with code that tests things more specifically, I'd be happy to run it :slightly_smiling_face: