Feature request: Configure zero backups

[kreempuff]

Have an idea to improve PGO? We'd love to hear it! We're going to need some information from you to learn more about your feature requests.

Please be sure you've done the following:

• [x] Provide a concise description of your feature request.
• [x] Describe your use case. Detail the problem you are trying to solve.
• [x] Describe how you envision that the feature would work.
• [x] Provide general information about your current PGO environment.

Overview

Make backups optional.

I get an error like this when I try to configure an empty backups section:

The PostgresCluster "example" is invalid: spec.backups.pgbackrest.repos: Invalid value: 0: spec.backups.pgbackrest.repos in body should have at least 1 items

Use Case

Simpler configuration of inconsequential database clusters. Configuring a PG cluster requires the configuring a backup strategy even if the backups won't be used. For example, for dev databases or for testing purposes.

Desired Behavior

The validator should allow objects to be created without any backups configured. Omitting at any level would be acceptable:

1. An empty repo list

   apiVersion: postgres-operator.crunchydata.com/v1beta1
   kind: PostgresCluster
   ...
   backups: 
   pgbackrest: 
     repos: []

2. An empty pgbackrest section

   apiVersion: postgres-operator.crunchydata.com/v1beta1
   kind: PostgresCluster
   ...
   backups: 
   pgbackrest: {}

3. An empty backups section

   apiVersion: postgres-operator.crunchydata.com/v1beta1
   kind: PostgresCluster
   ...
   backups: {} 

4. No backups section

   apiVersion: postgres-operator.crunchydata.com/v1beta1
   kind: PostgresCluster
   ...
   instances:
   ...
   postgresVersion: 13

   Environment

Tell us about your environment:

Please provide the following details:

EKS

• Platform: EKS
• Platform Version: 1.21
• PGO Image Tag: (e.g. ubi8-5.1.0-0)
• Postgres Version (e.g. 14)
• Storage: (e.g. hostpath, nfs, or the name of your storage class)
• Number of Postgres clusters: (XYZ)

Additional Information

Please provide any additional information that may be helpful.

[sagikazarmark]

I realize this is not really a good idea in production environments, but in dev backups often don't matter.

2531 might be related

[james-callahan]

See https://github.com/CrunchyData/postgres-operator/pull/2662#issuecomment-922616677

[darktempla]

Any movement on this one?

Another use case for no backups is if you are using something like longhorn storage or netapp astra etc then you would rely on them to do backups/restore of your persistent volume claims. Having the operator do this for us and having no option to disable is resulting in a waste of cluster resources (cpu/memory) and disk/storage space for zero benefit.

[benjaminjb]

Just dropping a quick note here to say that we have a feature in our backlog for disabling backups. This has been a feature requested a few times, so I'm trying to consolidate the issues:

https://github.com/CrunchyData/postgres-operator/issues/3432 https://github.com/CrunchyData/postgres-operator/issues/2531

[MarioMartReq]

hey @benjaminjb, are there any updates on this topic? I can't believe that I need to research how to fight against WAL/backups taking all of the storage space when I dont even need backups

[mkjpryor]

@benjaminjb

Is there any movement on this, or anything that can be done to help move it along?

Our use case is similar to the Longhorn case in that we are using Velero to back up all the other objects and PVCs in our cluster, so we have decided to also use it to back up the PVCs for our Postgres clusters. Because of the way we currently have Velero configured, this means that we are actually taking a backup of the pgbackrest volume containing backups that we never use, so the wasted storage is even worse!

I guess the only thing to resolve in the case when there are no backups is knowing when the WAL can be discarded.

[andrewlecuyer]

Hi @mkjpryor, thanks for reaching out about this!

You're timing is actually great, since we've currently got some plans in the works to take a closer look at this feature request. This need/request is therefore definitely on our radar, and we'll be following-up as more details, changes, etc. become available.

[benjaminjb]

Hi @mkjpryor, I just wanted to double-check my understanding of your use-case: you're using Velero to backup your K8s objects and PVCs.

To backup PVs, the Velero docs say that the backup operation "Calls the cloud provider API to make disk snapshots of persistent volumes, if specified."

Since I've not used Velero, I just wanted to check with you: do the Velero docs accurately describe what's happening? You're getting disk snapshots of your PVCs (both data and pgbackrest volumes)?

[mircea-pavel-anton]

I also have a need for this. In my case, I'm using CPGO as the backend for a Keycloak instance and I don't really need backups for it since all configuration is done via Terraform and IaC

Currently backing up to emptydir as a workaround

[andrewlecuyer]

I'm happy to report that support for optional backups has been added per PR https://github.com/CrunchyData/postgres-operator/pull/3977, and will therefore be available in the next release of CPK.

[dsessler7]

I'm pleased to announce that support for optional backups has been released with Crunchy Postgres for Kubernetes v5.7.0. Please see the docs for more info. If you get a chance to try it out, please let us know what you think!