Currently, all iterations of the benchmark for old commit run before the one for new commit. This can make the noise not evenly distributed across 2 benchmark runs (e.g. the machine heats up too much after running all iterations the old benchmark). This commit introduces interleave mode when running benchmark where each iteration of benchmark for old commit is followed by an iteration of benchmark for new one then back to an iteration of benchmark for old one.
Currently, all iterations of the benchmark for old commit run before the one for new commit. This can make the noise not evenly distributed across 2 benchmark runs (e.g. the machine heats up too much after running all iterations the old benchmark). This commit introduces interleave mode when running benchmark where each iteration of benchmark for old commit is followed by an iteration of benchmark for new one then back to an iteration of benchmark for old one.