Create a tool for local benchmarks

[sylwiaszunejko]

There is a need for a tool to do local benchmarks to make sure the performance is not regressing when adding new features, merging upstream etc.

[roydahan]

Let's create a document to design and brainstorming the approach of the tool to be used and the way we can test it pre-releasing the driver.

In this sprint we will start only with high level design and research.

[sylwiaszunejko]

@roydahan Just to be sure we are on the same page: @avelanarius made me realize that we can specify specific commit in scylla-bench to test it. I the purpose of this planned tool to run scylla-bench in CI w commit specified instead of release or do we want to create a brand new tool for this benchmarks?

[dkropachev]

We can record some traffic and inject it into Conn.r that would help us reproduce use cases to a T and have reliable benchmarks without any server or other tooling

[roydahan]

@roydahan Just to be sure we are on the same page:

@avelanarius made me realize that we can specify specific commit in scylla-bench to test it. I the purpose of this planned tool to run scylla-bench in CI w commit specified instead of release or do we want to create a brand new tool for this benchmarks?

We can use Scylla-bench, but I wanted to explore more ideas.

[roydahan]

We can record some traffic and inject it into Conn.r that would help us reproduce use cases to a T and have reliable benchmarks without any server or other tooling

Sounds interesting, can you elaborate or refer us to something similar?