Now that we have semi-reliable measurement and analysis, we need to provide an option for assessing potential downstream perf impact before a PR lands. I think the simplest way to implement this would be to allow specifying a list of additional toolchains to measure (each optionally with a URL to a rustup tarball?).
If we keep this file in the data directory, we can take PRs against https://github.com/anp/lolbench-data to add to the list of toolchain comparisons to include, and we can render comparison pages for each custom toolchain.
Now that we have semi-reliable measurement and analysis, we need to provide an option for assessing potential downstream perf impact before a PR lands. I think the simplest way to implement this would be to allow specifying a list of additional toolchains to measure (each optionally with a URL to a rustup tarball?).
If we keep this file in the data directory, we can take PRs against https://github.com/anp/lolbench-data to add to the list of toolchain comparisons to include, and we can render comparison pages for each custom toolchain.
This would enable one of the workflows requested in https://github.com/rust-lang/rfcs/issues/727.