Open eddyashton opened 1 month ago
I looks like bencher.dev may be what we want.
Test job: https://github.com/microsoft/CCF/blob/bencher_experiment/.github/workflows/bencher.yml, works on ubuntu 20.04 using a 1ES pool.
The json adapter supports both latency and throughput, and the format is very simple: https://bencher.dev/docs/explanation/adapters/#-json
I think we would move the performance tests to GHA for simplicity, and we would start with just running on main. The setup for PRs on branches seems to need a bit of care to avoid exposing secrets, but we may be able to do without it and just check periodically.
We disabled
cimetrics
in #6125, to avoid use ofMETRICS_MONGO_CONNECTION
.We should look at restoring
cimetrics
, either with improved auth to mongo, or by finding an alternative storage mechanism.