Whenever I run the bench bot on the pallet_contracts I need to go through the diff to see whether some benchmark result has substantially changed. And there are a lot of benchmarks.
It would be nice if the benchbot could analyze the diff for me and post a summary that contains:
Benchmarks that did substantially change their weight (like 50% or something)
New benchmarks
Removed benchmarks
What tools have you already tried
None. The problem isn't annoying me enough to do something about it.
How bad is the pain?
[x] Itching from time to time. I'm open for discussion
[ ] Fairly intense. I'd really like to have a solution soon
[ ] LITERALLY BLOCKER
Additional info
I realize that this is mostly a problem for contracts and that this might be of low priority. Still, I wanted to write it down somewhere.
The following report is from @athei
Tell us your story
Whenever I run the bench bot on the
pallet_contractsI need to go through the diff to see whether some benchmark result has substantially changed. And there are a lot of benchmarks.It would be nice if the benchbot could analyze the diff for me and post a summary that contains:
What tools have you already tried
None. The problem isn't annoying me enough to do something about it.
How bad is the pain?
Additional info
I realize that this is mostly a problem for contracts and that this might be of low priority. Still, I wanted to write it down somewhere.