Currently the test code seems set up for comparing different compilers. When making a pr potentially affecting compiler performance it is often asked to characterize the effect before deciding to merge it. Currently the way to do this would be to e.g. git checkout one version, run the benchmarks, check out another version, run the benchmarks, gather the results and write code to compare.
In the spirit of the CLI pr (#27) it might be nice to be able to do something like,
red-queen -c <compiler> --compare <commit id 1> <commit id 2>
Perhaps it's possible to do this with virtual environments, e.g. tox, so it doesn't touch your git repo?
Currently the test code seems set up for comparing different compilers. When making a pr potentially affecting compiler performance it is often asked to characterize the effect before deciding to merge it. Currently the way to do this would be to e.g. git checkout one version, run the benchmarks, check out another version, run the benchmarks, gather the results and write code to compare.
In the spirit of the CLI pr (#27) it might be nice to be able to do something like,
Perhaps it's possible to do this with virtual environments, e.g. tox, so it doesn't touch your git repo?