Open MahdiBM opened 2 months ago
It’s very intentional to not log everything - some benchmarks suites are quite large and it becomes very verbose -,the idea is to focus on deviations - maybe a verbose flag could be considered for that if it’s desired?
Ah right, I didn't mean that the default should necessarily change. An option would be enough.
AFAIK
benchmark check
only report the failed checks. I want it to always report everything. Even if nothing has failed. Perhaps the failed checks should be at the beginning of the report, and then the checks that passed.Currently we use
benchmark read
to read the results and attach them to a PR comment, but that doesn't contain the comparisons at all, so while the checks have passed, I can't immediately and easily compare the passed checks to the static thresholds that we're using.