Closed rukai closed 3 months ago
Comparing rukai:chain_benches_prerun
(2f5f934) with main
(911e8a0)
✅ 37
untouched benchmarks
🆕 2
new benchmarks
⁉️ 1
dropped benchmarks
:warning: _Please fix the performance issues or acknowledge them on CodSpeed._
Benchmark | main |
rukai:chain_benches_prerun |
Change | |
---|---|---|---|---|
⁉️ | query_counter |
13.8 µs | N/A | N/A |
🆕 | query_counter_fresh |
N/A | 14.1 µs | N/A |
🆕 | query_counter_pre_used |
N/A | 14 µs | N/A |
While assisting with https://github.com/shotover/shotover-proxy/issues/1649 I noticed that our chain benches are slightly inaccurate in that they run the against a fresh transform chain every time. In a more realistic scenario the chain is reused across requests and the transforms can cache values internally after processing requests that can be reused on the following request.
Due to the way criterion works we cant reuse the chain from the previous iteration of the bench. So the best way to solve this is to send an unmeasured request batch through the chain during setup. This ensures that each iteration of the bench will operate on a chain which has had exactly one request batch pass through it already.
For the query_counter bench it will be useful to measure in both ways, so I've split it into
query_counter_fresh
andquery_counter_pre_used
. Which use a fresh chain and a pre used chain respectively.For all other existing benches I converted them to use the pre used chain since that is most realistic.