search

babylonchain / finality-provider

A peripheral program run by the finality providers
Other
16 stars 28 forks source link

optimize e2e tests run time #346

Open bap2pecs opened 4 months ago

bap2pecs commented 4 months ago

it take too long to run those tests locally. each tests shouldn't exceed 10 sec imo

--- PASS: TestFinalityProviderLifeCycle (93.66s)
--- PASS: TestDoubleSigning (125.38s)
...
gitferry commented 4 months ago

It takes time to produce blocks, send finality votes, and wait for blocks to be finalized. Maybe we can reduce the block production time?

SebastianElvis commented 4 months ago

These are end-to-end tests where we run the real binaries and test each use case, so it might make sense to take more time. Nevertheless I do agree the time consumed could be reduced. There are several approaches:

  1. combine all tests into a test suite and run the tests sequentially on a single exec environment
  2. move the e2e tests into another CI pieline
  3. identify part of the code that consumes most time and think about how to reduce it

1/2 are immediately actionable and I recommend we do 2 like in Babylon repo (if we have bandwidth). 3 might need some invetigations though

bap2pecs commented 3 months ago

one way to optimize is to is change the number of PR from 1000 to 100 in e2e test

code:

randList, msgPub, err := GenCommitPubRandListMsg(r, fpSk, 1, 1000)
SebastianElvis commented 3 months ago

one way to optimize is to is change the number of PR from 1000 to 100 in e2e test

code:

randList, msgPub, err := GenCommitPubRandListMsg(r, fpSk, 1, 1000)

I'm not sure if this is the hotspot for the running time. Generating 1000 Merkle proofs shouldn't take many seconds but the test takes more than 100s.

bap2pecs commented 2 months ago

https://github.com/babylonchain/finality-provider/pull/503 adds parallelism and caching to the CI which cuts the time from 15 minutes to 6 minutes