Open hanno-becker opened 1 hour ago
@potsrevennil Do you know how to do this?
I'm thinking about using git note
although I've never used it before
~~For example, we could add summary of the benchmarking result (whenever there's a new commit on main) via git note
, which add notes to commit without touching commit itself, thus would not pollute the history.
Then we could just simply diff the notes between different commits on the main branch~~
But this might required some times to trial and error
Oh, maybe there's an easier way
the benchmarkaction/github-action-benchmark provided alert-threshold
and comment-on-alert
, this might be able to achieve what we want ?
@potsrevennil That sounds promising indeed!
It would be nice if the benchmarking CI could compare new vs.
main
performance numbers and warn, through a comment on the PR, if there's a target experiencing a major (>5%, say?) regression.