This PR adds a GH action that generates a historical timeline and a per-action run report of existing benchmarks on every merge to master.
We use free CI machines, so things can be somewhat noisy for activating automatic alerting without setting an absurd threshold of 40%. We can try to watch plots occasionally and see if this is useful.
I had to manually create a gh-pages branch to prepare a bit of the stage for the action to push the generated website to the GH pages automatically.
Context: I've discovered a gnark-crypto perf regression while experimenting with an idea in this repo while running benchmarks locally! I hope this automatic report can get us an earlier signal in a similar situation.
This PR adds a GH action that generates a historical timeline and a per-action run report of existing benchmarks on every merge to
master.We use free CI machines, so things can be somewhat noisy for activating automatic alerting without setting an absurd threshold of 40%. We can try to watch plots occasionally and see if this is useful.
I had to manually create a
gh-pagesbranch to prepare a bit of the stage for the action to push the generated website to the GH pages automatically.Context: I've discovered a gnark-crypto perf regression while experimenting with an idea in this repo while running benchmarks locally! I hope this automatic report can get us an earlier signal in a similar situation.