search

zalando / postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
https://postgres-operator.readthedocs.io/
MIT License
4.29k stars 974 forks source link

Are all patroni parameters supported? Use of pgbackrest for backup and restore. #1205

Open deepd opened 3 years ago

deepd commented 3 years ago

Hi

I'm looking to use pgbackrest along with the zalando postgres-operator. The use case is to take backups (full and incremental) and do a PITR when need be.

I have a few questions:

  1. I'd like to know if the operator supports all patroni parameters (for example 'archive_command' as shown here)? 1.1. If not, I can have another container that patches the patroni config at http://localhost:8008/config to update the parameter(s); Would that trigger a restart / reload of config in postgres? If not, how do you go request the operator to reload the configuration?
  2. Is there any plan to support pgbackrest natively in the operator in future?
  3. How do you request the operator to stop the database cluster so that pgbackrest restore can be executed and then request the operator to startup the database to do the restore automatically?

Thanks.

Edit1: I may be wrong, but I see https://github.com/zalando/postgres-operator/blob/v1.5.0/pkg/cluster/k8sres.go#L409 and https://github.com/zalando/postgres-operator/blob/v1.5.0/pkg/cluster/sync.go#L440 and feel like not all patroni parameters are supported. - IGNORE

Edit2: It looks like I can just add an "archive_command" under spec.postgresql.parameters in the manifest for a database cluster and operator handles it.

Jan-M commented 3 years ago

I have not looked into pgbackrest, wal-g and wal-e seem to do their job. Ultimately I would think you need to solve this in your docker image. The operator has ample ways to inject gcp/aws credentials and destination path, so you should be able to solve this. I am not sure if Patroni supports it, so this sounds like a bigger issue, but also something potentially unnecessary.

I do not see us invest into this, neither in Spilo nor in the Operator.

Samusername commented 1 year ago

Would you (developers) reconsider possibility to run/use pgBackRest with zalando postgres-operator?

Usage of S3 buckets / wal-e/wal-g sounds a little complex setup, for using/running such in deliveries in multiple separate systems. And, disk space consumption in a S3 bucket?

( Or outside postgres-operator: it might be possible to restore leader from volume snapshot. Some day. Etc. )

Or, is it easy to just to try to use pgBackRest in archive_command / and restore command, etc: "It looks like I can just add an "archive_command" under spec.postgresql.parameters in the manifest for a database cluster and operator handles it." ( E.g. by Leader. Etc. Somehow. )