search

foundry-rs / foundry

Foundry is a blazing fast, portable and modular toolkit for Ethereum application development written in Rust.
https://getfoundry.sh
Apache License 2.0
8.31k stars 1.75k forks source link

bug: state appears to be shared between tests when linked libraries are used #8639

Open mds1 opened 3 months ago

mds1 commented 3 months ago

Component

Forge

Have you ensured that all of these are up to date?

What version of Foundry are you on?

forge 0.2.0 (1197fbe 2024-08-09T00:25:08.996768000Z)

What command(s) is the bug in?

forge test

Operating System

None

Describe the bug

  1. Clone the optimism monorepo and check out this branch: https://github.com/ethereum-optimism/optimism/pull/11423
  2. Run just test, and you'll notice testDebug fails. That is hardcoded data from a failed CI run
  3. Now, only run the failed test: forge test --mt testDebug -vvv. It will pass. If you run the full file containing that test it will also still pass.
  4. Compare the passing and failing traces (see below screenshots). Notice how in the passing one, the call is to an unlabeled address, and in the failing one it's to an address labeled ForgeArtifacts. Based on the hardcoded calldata in testDebug, we can see that address is 0x5F65cD7D792E9746EF82929D60de9a1C526f93A5.

So what appears to be happening here is that when the full suite is run, contracts deployed from another test (1) leak into the fuzzer dictionary of this test, and (2) the state leaks into this test as well.

image image
mattsse commented 3 months ago

@grandizzy I wonder if the dictionary is unique per test contract and not TestContract::test ?

mds1 commented 3 months ago

@mattsse I don't think so because forge test --mp test/libraries/SafeCall.t.sol also passes

It seems likely the issue is actually related to library deployments and linking: https://github.com/ethereum-optimism/optimism/pull/11426#issuecomment-2278508861. Once I remove all public lib methods in https://github.com/ethereum-optimism/optimism/pull/11426, the test passes

grandizzy commented 3 months ago

@mattsse I actually think there's a bug introduced with https://github.com/foundry-rs/foundry/pull/8497 here, as it seems the executor for fuzz tests is not cloned anymore? https://github.com/foundry-rs/foundry/commit/4a41367398ddc35faf705f5d93e7e1a4eae3884d#diff-98c09a5d1cb595f37109d7b1c0109ee32dd74dc1d253cdda9a63e4ce32bf49d5R628-R671

Le: actually Cow::Borrowed().into_owned() is cloning it, and rechecked test is failing even with that feature reverted, so probably not this

mds1 commented 3 months ago

@mattsse I updated my above comment, but have confirmed the cause of the bug was library linking: once we remove all public methods from libs the issue goes away

klkvr commented 3 months ago

Since https://github.com/foundry-rs/foundry/pull/8034 when running test suites requiring library linking, forge predeploys all libraries which are required by at least one test contract and reuses resulted state. This likely causes some of those libraries being used as fuzz dictionary values, which could've resulted in this failure