Google Test has INSTANTIATE_TEST_SUITE_P to specify parameters along multiple dimensions and automatically generate the various combinations. Google Benchmark seems to do this via "ranges": https://github.com/google/benchmark#passing-arguments.
Anticipating the number of upcoming combinations, maybe it is worth to look into these mechanisms.
If we do not want to execute the entire resulting matrix, consider https://github.com/hyrise/hyrise/wiki/Input-Domain-Modeling and just "document" properly in a structured way which parameters we benchmark in which order.
Feedback from meeting from 2019/12/18: Let's try to get rid of the fixtures for the sort micro benchmarks, maybe by using only one fixture and the rest of the benchmarks as "normal" fixtures.
Google Test has INSTANTIATE_TEST_SUITE_P to specify parameters along multiple dimensions and automatically generate the various combinations. Google Benchmark seems to do this via "ranges": https://github.com/google/benchmark#passing-arguments.
Anticipating the number of upcoming combinations, maybe it is worth to look into these mechanisms. If we do not want to execute the entire resulting matrix, consider https://github.com/hyrise/hyrise/wiki/Input-Domain-Modeling and just "document" properly in a structured way which parameters we benchmark in which order.
Originally posted by @tobodner in https://github.com/bakoe/hyrise/pull/2