search

haskell / criterion

A powerful but simple library for measuring the performance of Haskell code.
http://www.serpentine.com/criterion
BSD 2-Clause "Simplified" License
503 stars 86 forks source link

Allow 2D grouping #283

Closed anka-213 closed 5 months ago

anka-213 commented 5 months ago

I have a situation where I am comparing things along two dimensions. More specifically, I am benchmarking multiple implementations of the same code on a bunch of different examples separately.

What I would like to do is keep the colors of the top level groups for the different implementations, while sorting the output in the graph so that all the benchmarks for the same example are next to each other

Alg1 / Ex1   -- Colored yellow
Alg2 / Ex1  -- Colored light blue
Alg1 / Ex2  -- Colored yellow
Alg2 / Ex2 -- Colored light blue
...

Because the name of the group is prepended to the name of the individual testcase, lexical sorting doesn't do what I want. And naming all the groups the same seems to make them treated as just the same testcase, so that doesn't work either

anka-213 commented 5 months ago

Wait, Colexical already does what I want, I had missed that!

anka-213 commented 5 months ago

Though, it would be very nice if I could sort the examples by time, while still keeping all the instances of the same example together. For example, I would sort by the time it takes for Alg2 and keep the Alg1 for the same example right above it, regardless of how long time Alg1 took for that example.

Alg1 / SlowestEx
Alg2 / SlowestEx
Alg1 / FasterEx
Alg1 / FasterEx
...