Open potsrevennil opened 4 days ago
Okay, we have to resolve the variance first before proceeding with the alerts. I opened an issue for that: https://github.com/pq-code-package/mlkem-native/issues/429
Turning this into a draft PR for now.
Variance should be fixed now that we have moved c7i to a metal instance. @potsrevennil Can you update this PR?
Resolves #223
The alert threshold for performance regression is set to 103% for now, let me know if there's any better suggestion for the threshold.
If the threshold is exceeded, the bench bot will automatically leave a comment on PR like this:
https://github.com/potsrevennil/mlkem-c-aarch64/pull/27
There will also be GH job summary generated.
There's some other options provided by the benchmark action (but not set in this pr) which one might be useful as well: