Fix: Disable logging in benchmarks

fknorr commented 6 months ago

Note: This PR is based on #248.

In #234, we set the global log level to trace in all tests in order to print a full log in every failing test. Since we did not perform a benchmark run, we did not notice its impact on our benchmarking suite (see the benchmark report in #248): Task-graph generation emits trace-logs whenever testing for horizon generation, which is triggered for every command group submission. These logs are being written to the test log capture buffer which is very expensive compared to the actual task generation.

This PR explicitly shuts off all logging for the duration of any benchmark, re-activating the behavior that was implicitly true before #248 (because benchmarks were run with the default LOG_LEVEL).

I have re-run the benchmarks to commit a new baseline.

github-actions[bot] commented 6 months ago

Check-perf-impact results: (f7ef1830fe7f830eee417b0b5a37c2ef)

:warning: Significant slowdown (>1.25x) in some microbenchmark results: building command graphs in a dedicated scheduler thread for N nodes - 1 > immediate submission to a scheduler thread / contracting tree topology
:rocket: Significant speedup (<0.80x) in some microbenchmark results: building command graphs in a dedicated scheduler thread for N nodes - 4 > immediate submission to a scheduler thread / contracting tree topology

Relative execution time per category: (mean of relative medians)