search

readysettech / readyset

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.
https://readyset.io
Other
4.26k stars 117 forks source link

Clickhouse Support #29

Open tomtom215 opened 1 year ago

tomtom215 commented 1 year ago

Is your feature request related to a problem? Please describe.

Support for Clickhouse DB

Additional context

With Clickhouse being primarily a large data OLAP DB, integration with it could lead to a wider adoption of ReadySet

jbensal commented 1 year ago

Hi @tomtom215!

Thanks so much for your request. Can you elaborate a bit more on how you're using Clickhouse and how you want to use it with ReadySet?

We currently don't have Clickhouse support on our roadmap as we're focused on being the fastest caching layer for Postgres and MySQL, but I can let you know if that changes.

tomtom215 commented 1 year ago

Hi @jbensal Thanks for your reply. We are using Clickhouse as a central OLAP DB in our data lake due to it's speed of ingestion. The usecase where I could ReadySet is that Clickhouse is not the best to have 100+ end users directly querying the database concurrently.

ReadySet would allow us to independently scale the Clickhouse cluster as needed for ingestion needs not worry about the impact of direct end-user queries. For example, if we generate a daily dataset to be used by 10K employees for a dashboard that is powered by dozens of queries, Clickhouse would get hammered when multiple people refresh that dashboard at the same time, whereas with ReadySet in between as a caching layer, we would reduce the DB load and improve performance & response times for everyone.

alexey-milovidov commented 1 year ago

ClickHouse added the support for query cache: https://clickhouse.com/blog/introduction-to-the-clickhouse-query-cache-and-design

It is not an automatically updatable view as ReadySet, but fits your scenario.

tomtom215 commented 1 year ago

Wow, I was not expecting a response from the legendary @alexey-milovidov ! Thank you for letting me know about ClickHouse's new query caching, we'll explore it right away!

@jbensal - let me know if you would like to close this issue. I still think there's a lot of value in supporting ClickHouse, but I understand if you feel that Alexey's suggestion is a better architecture approach for the near term.