search

matter-labs / foundry-zksync

Fork of Foundry tailored for zkSync environment
Apache License 2.0
291 stars 125 forks source link

Bug Report: Slow compile time for large Foundry projects #378

Closed dutterbutter closed 1 month ago

dutterbutter commented 4 months ago

Description

Compile time for Foundry projects is significantly longer due to the necessity of compiling tests and scripts in addition to the main project files. Initial improvements have been made in this pull request, but repositories with a substantial number of Solidity files still experience lengthy compilation times.

NOTE: A larger refactor of the compiler logic in foundry-zksync is currently on-going in an effort to prepare its inclusion in foundry-compilers. That worked can be observed here, and here.

Steps to Reproduce

  1. Ensure the latest foundry-zksync forge is installed by following the installation guide here.
  2. Clone a large repository to compile: 
    git clone https://github.com/bgd-labs/aave-v3-origin
  3. Ensure you are not using solc 0.8.25 as the default solc compiler, as the current version of foundry-compilers used in foundry-zksync does not support it. Refer to this issue for more information.
  4. Run the following command: 
    RUST_LOG=debug forge build --zksync
  5. Observe that the compilation process takes a long time to complete.

Expected Behaviour

The compilation process should be more efficient, reducing the overall compile time. The compilation is slightly different then hardhat projects, as in Foundry projects everything is written in Solidity (tests, deployment scripts etc) so they all need to be compiled.

Relevant Points

Additional Information

hedgar2017 commented 4 months ago

Thanks for the report!

  1. Could you guys verify that the problem is in zksolc, e.g. by measuring the execution time?
  2. Does Foundry have some form of incremental compilation?
dutterbutter commented 4 months ago

@hedgar2017 any chance you can assist in measuring the execution time? Not sure the best approach to do so. As for the incremental compilation can give insight into exactly that means?

In the improvement PR linked, the way we invoke the compiler was a big area of improvement that is, where as before we'd invoke the compiler separately for each contract (with its own set of dependencies), we now call the compiler a limited number of times (typically 1) with all the files in the project and the dependencies.

hedgar2017 commented 4 months ago

@dutterbutter isn't it possible to measure how much time zksolc takes to execute from Foundry, and compare it to the total time?

zksolc must be called once for the whole project, passing single standard JSON input. hardhat does that so you can reverse-engineer it.

Karrq commented 4 months ago

@hedgar2017 The summary provided by @dutterbutter is accurate.

During the optimization work, providing the entire set of sources to zksolc brought the most improvement, and that is what we are going to do moving forward (from #377). This brought the time down from about 4hrs to 36m of total runtime.

We have optimized some overhead that we add on top (preparing the input, parsing the output, storing artifacts to disk), but nevertheless there's a major difference in runtime when tests and scripts contracts are compiled.

I'll provide more accurate timings soon (time waiting for the compiler output vs total runtime), but the difference in number of contracts is rather small (74 out of 336), whereas the total runtime increases by a factor of x24 (1m30s to 36m)

hedgar2017 commented 4 months ago

@Karrq thanks! Looking forward to the timings!

Karrq commented 4 months ago

I shared a spreadsheet with the timings via internal channel, so I'll share them again here for consitency

Timings spreadsheet: https://docs.google.com/spreadsheets/d/17E1l5ryxF2tFEYWxHwX9CLEHHHgB03BurWjtjCIxDS8/edit?usp=sharing

hedgar2017 commented 4 months ago

I shared a spreadsheet with the timings via internal channel, so I'll share them again here for consitency

Timings spreadsheet: https://docs.google.com/spreadsheets/d/17E1l5ryxF2tFEYWxHwX9CLEHHHgB03BurWjtjCIxDS8/edit?usp=sharing

Thanks! Internally we've confirmed that it's on the solc side if viaIR is used. We'll forward that to the solc developers.

Karrq commented 4 months ago

We have merged the bigger compiler refactor effort and have gained quite a lot of time. We also discussed internally and came to a few more conclusions on the slow compilation - we'll be waiting for the next zksolc compiler release (and ZkSync's solc) which will further reduce compilation time

Karrq commented 1 month ago

zksolc 1.5.1 improved compile times greatly, along with solc 0.8.26 🚀