Split system_tests into multiple suites such that each of them can finish CICD testing in time

AntelopeIO / reference-contracts

Other

10 stars 14 forks source link

Split system_tests into multiple suites such that each of them can finish CICD testing in time #100

Closed linh2931 closed 4 months ago

linh2931 commented 4 months ago

Resolved https://github.com/AntelopeIO/reference-contracts/issues/99.

Fixed test failure in Spring https://github.com/AntelopeIO/spring/issues/254

Change Description

Deployment Changes

[ ] Deployment Changes

API Changes

[ ] API Changes

Documentation Additions

[ ] Documentation Additions

spoonincode commented 4 months ago

What caused the ballooning of system_tests execution time? It's nearly 7x from what it was before switching to the new testers in #95. I'm not surprised to see something like 2x but 7x is a bit surprising considering it doesn't look like we added any new tests or such in #95.

linh2931 commented 4 months ago

What caused the ballooning of system_tests execution time? It's nearly 7x from what it was before switching to the new testers in #95. I'm not surprised to see something like 2x but 7x is a bit surprising considering it doesn't look like we added any new tests or such in #95.

I am digging it deeper on the cause. Thanks.

linh2931 commented 4 months ago

Initial numbers when running eosio_system_tests with time command on my Ubuntu22 box:

Legacy tester: 6m32.055s
Savanna tester with 4 local BLS keys (current configuration): 18m19.707s
Savanna tester with 1 local BLS key: 13m31.346s

Looking into why even 1 key doubles the test time.

linh2931 commented 4 months ago

Configuring only one local finalizer helps to reduce system_tests to run in 7 minutes in CICD https://github.com/AntelopeIO/spring/pull/266.

Close this PR for now.