microsoft / CCF

Confidential Consortium Framework
https://microsoft.github.io/CCF/
Apache License 2.0
761 stars 205 forks source link

Find a way to restore or replace cimetrics #6126

Open eddyashton opened 1 month ago

eddyashton commented 1 month ago

We disabled cimetrics in #6125, to avoid use of METRICS_MONGO_CONNECTION.

We should look at restoring cimetrics, either with improved auth to mongo, or by finding an alternative storage mechanism.

achamayou commented 1 week ago

I looks like bencher.dev may be what we want.

Test job: https://github.com/microsoft/CCF/blob/bencher_experiment/.github/workflows/bencher.yml, works on ubuntu 20.04 using a 1ES pool.

The json adapter supports both latency and throughput, and the format is very simple: https://bencher.dev/docs/explanation/adapters/#-json

I think we would move the performance tests to GHA for simplicity, and we would start with just running on main. The setup for PRs on branches seems to need a bit of care to avoid exposing secrets, but we may be able to do without it and just check periodically.

achamayou commented 1 week ago

pi_basic_virtual_cft