search

Altinity / clickhouse-operator

Altinity Kubernetes Operator for ClickHouse creates, configures and manages ClickHouse clusters running on Kubernetes
https://altinity.com
Apache License 2.0
1.82k stars 449 forks source link

feature request: support autoscaling ClickHouse cluster #1237

Open prashant-shahi opened 10 months ago

prashant-shahi commented 10 months ago

It would be really useful to support autoscaling clickhouse cluster based on some events or metrics similar to how KEDA does.

Slach commented 10 months ago

do you mean vertical auto scaling or horizontal autoscaling?

prashant-shahi commented 10 months ago

do you mean vertical auto scaling or horizontal autoscaling?

@Slach For now, what I had in mind was the horizontal autoscaling.

Slach commented 10 months ago

horizontal autoscaling during adding more shards have no sense, cause ClickHouse itself doesn't support data re-sharding

horizontal autoscaling for adding new replica in exists shard, have non-clean criteria for autoscaling

currently we think about autoscaling PVC size for disk, but it has restrictions, no more 1 resize in 6 hour for AWS, for example

jirislav commented 4 months ago

ClickHouse Cloud supports vertical autoscaling, maybe the operator could also support it?

However, it might be quite challenging as the enormous workload size often requires the nodes in k8s to be dedicated and their provisioning might require very complex integration with Openstack or AWS Node Groups.

tomershafir commented 3 months ago

I wonder whats the status here