It splits all the era-compiler-tester tests for PRs into directories/files/file+mode sets to allow for parallelization.
Uses system rocksdb to speed up compilation.
Makes shallow clones of submodules to speed up checkout.
Classifies and sorts passing and failing tests. All failing tests appear first.
Future work:
Document the process to update the test suite. It requires checking for new and deleted files in the era-compiler-tests/tests tree.
Document the process to allow passing tests. It should suffice to uncomment the respective line.
Try to compile the tester only once and reuse the binary to avoid wasting compute time.
Group better the tests, some are rather quick and could be executed together, while some others could take less time.
Try to improve caching. rust-cache doesn't seem to persist the dependencies, downloading each time, and maybe the system contracts could be cached as well.
Check if using the optimization level for the middle-end (M*) as second variable for everything makes jobs fast enough to reduce how many partitions we make. Maybe running one solidity/simple job per optimization level gives us good enough times.
Overall, this approach allows running the whole suite in under 17 minutes, which should be good enough for development.
Other notes:
compiler-tester provides options to save and load the system contacts. This can be used to cache them, saving about 20s per job. Something similar could be done to the several compilers the tool downloads.
This PR makes a handful of changes:
era-compiler-testertests for PRs into directories/files/file+mode sets to allow for parallelization.Future work:
era-compiler-tests/teststree.rust-cachedoesn't seem to persist the dependencies, downloading each time, and maybe the system contracts could be cached as well.solidity/simplejob per optimization level gives us good enough times.Overall, this approach allows running the whole suite in under 17 minutes, which should be good enough for development.
Other notes:
compiler-testerprovides options to save and load the system contacts. This can be used to cache them, saving about 20s per job. Something similar could be done to the several compilers the tool downloads.