search

AntelopeIO / leap

C++ implementation of the Antelope protocol
Other
113 stars 69 forks source link

IF: workflows fail because of newly added tests in `hotstuff_integration` branch #2369

Closed greg7mdp closed 2 months ago

greg7mdp commented 2 months ago

We determined that the addition of two new tests in https://github.com/AntelopeIO/leap/pull/2335 caused a string of workflow failures, causing at least one failure for just about every commit. Failures are in NP, LR, ubsan, asan, asserton workflows.

Two workflow runs, using hotstuff_integration with just these tests removed, passed without problems. See https://github.com/AntelopeIO/leap/actions/runs/8523025337.

Since we need the new tests in order to run validations both with and without Savanna activated, we need to beef-up the instances the workflow VMs are running on.

We determined that, even before the systematic failures, the tests were running extremely slowly.

In order to provide reliable ci/cd workflows, and to provide some buffer so that the issue doesn't reoccur as soon as we add a few new tests, I feel that we should at least double the number of cores per instance (or the equivalent if each core is faster), provided the cost is not prohibitive.

This increase should apply to the following workflows: NP, LR, ubsan, asan, asserton

arhag commented 2 months ago

Seems to be resolved by increasing performance of CI/CD VMs.