search

etcd-io / etcd

Distributed reliable key-value store for the most critical data of a distributed system
https://etcd.io
Apache License 2.0
47.35k stars 9.72k forks source link

Add Prow job to run periodic performance benchmarks #17754

Closed ivanvc closed 6 days ago

ivanvc commented 5 months ago

What would you like to be added?

As part of #16467, create a Prow job that nightly runs the rw-benchmarks. Storing the results as an artifact so results can be processed later and have an idea of how the development is affecting the project's performance.

Why is this needed?

To start collecting performance benchmarks to establish a project perspective on performance.

jmhbnz commented 5 months ago

Added to our tracking under https://github.com/etcd-io/etcd/issues/16467.

ivanvc commented 4 months ago

The first five performance jobs are running in the K8s test EKS clusters. We still need to add the last three jobs.

serathius commented 4 months ago

Thanks for integrating benchmarks into the prow, however I was hoping that we will do the work in semi order described in https://github.com/etcd-io/etcd/issues/16467#issuecomment-1691326907, so before we start running periodic benchmark we agree on what to measure first.

Having the rw-benchmark running on prow is good, however I would want to avoid a case where we later decide that we need different approach and throw out your work.

ivanvc commented 4 months ago

This makes sense, @serathius; thanks for your input. @jmhbnz, should we regroup and formalize a proposal?

jmhbnz commented 4 months ago

This makes sense, @serathius; thanks for your input. @jmhbnz, should we regroup and formalize a proposal?

Certainly. I am currently taking some time off to move house but will be returning to open source work 6th May and very keen to regroup when I return. Feel free to update https://github.com/etcd-io/etcd/issues/16467 with any thoughts in the interim and thanks again for your work getting these initial tests of prow for heatmaps running @ivanvc