search

zalando / postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
https://postgres-operator.readthedocs.io/
MIT License
4.24k stars 968 forks source link

Scaling in k8s cloud. #1469

Open Menzorg opened 3 years ago

Menzorg commented 3 years ago

Hello. Thanks a lot for this pretty operator.

I could not find info about scaling pods with a postgres operator! I have operator in my k8s cluster and CRD postgres, configured to be 3-instancies pg-cluster. So, if I want to enable autoscaling, what shall i do? Is it built in? I meen to auto-scale statefulset pod count.

I know, that k8s have a HPA for example, but... postgres operator has some checking feature to "repare" pg-cluster and if i change statefulset pods count manualy, then operator will "fix" to mainfest default value = 3. What do i need to do? Create pg-cluster with some other way that crd? Or maybe some patrony configuration? Maybe will find answer soon and will close issue, but i think, it may be question that interests not only me.

best regards, Piter.

Sicaine commented 3 years ago

What is your expectation on autoscaling though?

For a database setup, autoscaling up and down doesn't make much sense as it takes time and resources from your existing databases due to your new instance syncing up.

For bigger databases, that can take hours or days.

Menzorg commented 3 years ago

What is your expectation on autoscaling though? ... For bigger databases, that can take hours or days.

So, if i understood correctly, it is not inbox functional. My expectation is using HPA. Our DB is small (about 300mb), and it will not be much bigger for a long time. We have configured in crd count of instances, but we want to increase the number not manually when we have highload. In our case, when we increase count manually - every new instance starts for minute. To my mind it is good solution to have autoscaling ability in this scenario.

Menzorg commented 3 years ago

I think, it may be good solution, if we could tell confguration checker to check or not to check number of instancies, for example

apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
  name: test-pg-cluster
  namespace: test-namespace
spec:
  dockerImage: registry.opensource.zalan.do/acid/spilo-13:2.0-p6
  teamId: "test"
  numberOfInstances: 3
  checkNumberOfInstances: true

So, when it is false - operator will not force change statefullSet replica count. Then we can create HPA with some logic. Maybe usefull will be to more update postgresql CRD to configure HPA, if we want to have ability to create it with creating pg cluster.

sarmadali20 commented 2 years ago

@Menzorg this open will be helpful as I have read-only replicas and sometimes I want them to scale up based on the load.