search

ordo-one / package-benchmark

Swift benchmark runner with many performance metrics and great CI support
Apache License 2.0
326 stars 25 forks source link

Support absolute/relative thresholds deviation configurations for --check-absolute too #220

Closed hassila closed 10 months ago

hassila commented 10 months ago

Some memory notes:

I guess in your run now, the current run would be something like "mallocCountTotal" : 163633

such that 163644-164434 = -801 ~-1K

12:19:48 ========================================================================
12:19:48 Absolute threshold deviations for xxx:yyy
12:19:48 ========================================================================
12:19:48 ╒══════════════════════════════════════════╤═════════════════╤═════════════════╤═════════════════╕
12:19:48 │ Malloc (total) (K, Δ)                    │       Threshold │     Current_run │   Threshold Abs │
12:19:48 ╞══════════════════════════════════════════╪═════════════════╪═════════════════╪═════════════════╡
12:19:48 │ p90                                      │             164 │             164 │              -1 │
12:19:48 ╘══════════════════════════════════════════╧═════════════════╧═════════════════╧═════════════════╛
12:19:48 
12:19:48 New baseline 'Current_run' is BETTER than the defined absolute baseline thresholds. (--check-absolute)