In working on #128 , I ran some performance tests and noticed that log-level had a significant impact. logrus has been observed to be slow in go-algorand and there is an internal issue (2479) to replace it and a poc branch using zerolog.
to bootstrap testnet and run a postgresql exporter against it for 300 seconds. I ran it a number of times against both the original pipeline and the new one. Here are the experimental results:
Log Level
Reps
Original rounds/300 sec (logs/round)
Pipelining rounds/300 sec (logs/round)
Pipelining v Original (%)
TRACE
3
3718 (7.0)
3509 (14.0)
-5.6%
INFO
2
4578.5 (3.0)
4423.5 (3.0)
-3.4%
So comparing the results within each column we can see:
19% performance decline in the Original algorithm when going from 3 → 7 logs/round
21% performance decline in the Pipelining algorithm when going from 3 → 14 logs/round
The sample was very noisy but it looks like each log per round is costing around 1-5 % hit in terms of performance.
Action Items
A POC PR which let's us validate the performance implications of swapping out logrus for zerolog
Decide if we want to allow "plug and play" logger capability (we have heard one community member make this suggestion)
Switch all logs in conduit to the new logger (possibly via an interface if we opt for the "plug and play" approach)
Problem
In working on #128 , I ran some performance tests and noticed that log-level had a significant impact. logrus has been observed to be slow in go-algorand and there is an internal issue (2479) to replace it and a poc branch using zerolog.
Problematic experiment
Using the Justfile command
to bootstrap testnet and run a postgresql exporter against it for 300 seconds. I ran it a number of times against both the original pipeline and the new one. Here are the experimental results:
So comparing the results within each column we can see:
The sample was very noisy but it looks like each log per round is costing around 1-5 % hit in terms of performance.
Action Items
More links
Dependencies
None
Urgency
Medium - as we're currently working on improving Conduit's performance, this seems like a useful avenue to persue.