search

scylladb / scylla-operator

The Kubernetes Operator for ScyllaDB
https://operator.docs.scylladb.com/
Apache License 2.0
324 stars 159 forks source link

Support horizontal autoscaling #80

Open dahankzter opened 4 years ago

dahankzter commented 4 years ago

We should enable and use the "Horizontal Pod Autoscaler" (HPA) to support automatic scale up and downs. We have to make sure there is a latency in the behavior so there can be a smooth and controlled scale up and down. We don't want it to jump up and down...

MeydanOzeri commented 2 years ago

any update on that ?

dorlaor commented 2 years ago

There is an external autoscale project that works, you can try it out now

On Wed, Aug 11, 2021 at 10:26 AM Meydan Ozeri @.***> wrote:

any update on that ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/scylladb/scylla-operator/issues/80#issuecomment-897012053, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANHURPMWFRRTYCOBAII34DT4KXERANCNFSM4MZQUVYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

tnozicka commented 2 years ago

We had some discussions about autoscaling direction this week (centered around upstream HPA), @rzetelskik is looking into options and will be working a proposal that will land in https://github.com/scylladb/scylla-operator/tree/master/enhancements/proposals

(the student project Dor is referencing is https://github.com/scylladb/scylla-cluster-autoscaler)

mariusz-gomse-centra commented 4 months ago

Hello - is there any update on that?

I've got one proposition - maybe instead of 1 CRD which is ScyllaCluster we can have 2 or 3 CRDs:

ScyllaCluster will have then definition like:

...
spec:
  datacenter:
    name: "simple-datacenter" # it can be also get through labels/selectors 
...

ScyllaClusterDatacenter will be:

metadata:
  name: simple-datacenter
spec:
  racks:
    - name: "us-east-1a"  # it can be also get through labels/selectors

ScyllaClusterRack will be:

metadata:
  name: "us-east-1a"
spec:
  members: 3
  ...

Thx to that we will be able to use k8s scale subresource on the ScyllaClusterRack and instead of editing the CR we will be able to do kubectl scale --replicas=4 scyllaclusterrack/us-east-1a

Scale subresource can look like this in the

subresources:
        scale:
          specReplicasPath: .spec.members
          statusReplicasPath: .status.members

Thanks to this for autoscaling later can be use some external tool like KEDA

What do you think about it?

scylla-operator-bot[bot] commented 1 day ago

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

/lifecycle stale