We recently had a check run posted that was triggered by @ursabot please benchmark lang=C++.
Since the ArrowAlerter separates all benchmark results into "stable" and "unstable", and reports on only the "stable" ones, and we treat C++ benchmarks as "unstable", this check run resulted in ACTION_REQUIRED because there were no "stable" results.
Ideally we probably should report on benchmarks that were explicitly asked for, even if they're unstable.
We recently had a check run posted that was triggered by
@ursabot please benchmark lang=C++.Since the ArrowAlerter separates all benchmark results into "stable" and "unstable", and reports on only the "stable" ones, and we treat C++ benchmarks as "unstable", this check run resulted in ACTION_REQUIRED because there were no "stable" results.
Ideally we probably should report on benchmarks that were explicitly asked for, even if they're unstable.