The purpose of this PR is to optimize the performance of test generation and execution with the purpose of improving work quality on future changes.
The ssv module is the one that takes the most time. After running a CPU profiling test, we noticed that the NewTestingKeyManager function consumes 80% of the total time, which is explained by the fact that it's called multiple times and takes 26 milliseconds to run (according to benchmark tests).
Nonetheless, the NewTestingKeyManager function returns a testingKeyManager object that is only used to validate and sign data but does not change state throughout a test execution. Thus, we propose adding a singleton pattern for this object.
Metrics
To illustrate the number of times the NewTestingKeyManager function is called, take a look at the following table
Note: the number after the dash (/) is the number of different instances created, which depends on a slashableDataRoots parameter.
Module
Test generation
Test execution
QBFT
368/1
390/1
SSV
17058/1
20470/2
Types
48/1
48/1
Performance comparison
With the singleton pattern, if the appropriate object already has been created, the function takes 5 microseconds, rather than 26 milliseconds.
All time values shown below are in seconds (or in minutes:seconds if it has ":" ).
For test generation, we got the following performance results.
QBFT
SSV
Types
Old
15
7:36
3
New
5
20
3
For test execution, we got the following performance results.
Overview
The purpose of this PR is to optimize the performance of test generation and execution with the purpose of improving work quality on future changes.
The
ssv
module is the one that takes the most time. After running a CPU profiling test, we noticed that theNewTestingKeyManager
function consumes 80% of the total time, which is explained by the fact that it's called multiple times and takes 26 milliseconds to run (according to benchmark tests).Nonetheless, the
NewTestingKeyManager
function returns atestingKeyManager
object that is only used to validate and sign data but does not change state throughout a test execution. Thus, we propose adding a singleton pattern for this object.Metrics
To illustrate the number of times the
NewTestingKeyManager
function is called, take a look at the following tableNote: the number after the dash (/) is the number of different instances created, which depends on a
slashableDataRoots
parameter.Performance comparison
With the singleton pattern, if the appropriate object already has been created, the function takes 5 microseconds, rather than 26 milliseconds.
All time values shown below are in seconds (or in minutes:seconds if it has ":" ).
For test generation, we got the following performance results.
For test execution, we got the following performance results.