cmu-db / benchbase

Multi-DBMS SQL Benchmarking Framework via JDBC

https://db.cs.cmu.edu/projects/benchbase/

Other

428 stars 174 forks source link

Consider adding ClickBench #535

Open alexey-milovidov opened 1 month ago

alexey-milovidov commented 1 month ago

https://github.com/ClickHouse/ClickBench

The main advantage of ClickBench is that it has a good balance between being simple (this is how it has >50 DBMS) and still stressful enough (it is used not only for DBMS, but for hardware tests, as the tests max out the CPU, memory bandwidth, and IO).

Note: I'm the author of ClickBench (but it's doubtful whether I will have time to contribute it directly here, so it is just a proposal).

bpkroth commented 1 month ago

Happy to take a PR :)

Main things we would need are

[ ] driver updates to pom.xml
[ ] sample configs. See configs for some examples.
[ ] tests
- [ ] For CI see .github/workflows/maven.yml
- [ ] docker image docker/ (there are also scripts in here to help run and test things locally)

379 should be a decent PR to look at for other examples.

bpkroth commented 1 month ago

@alexey-milovidov, btw, if you don't have time to do the PR, but could at least comment with a few pointers on any of the above it could help speed things along for whoever does pick this up. Thanks!

alexey-milovidov commented 1 month ago

Sure, I will be happy to help.

apavlo commented 1 month ago

@alexey-milovidov What is the goal of this? The more benchmarks the better, but there's no way we're going to have the breadth of supported systems that you have.

bpkroth commented 1 month ago

Ah, good point. Didn't read closely enough and initially interpreted this as adding ClickHouse support.

alexey-milovidov commented 1 month ago

There is no goal at all. I see this project as a benchmark aggregator, and adding a fairly good benchmark will make it more complete.