k8up-io / k8up

Kubernetes and OpenShift Backup Operator
https://k8up.io/
Apache License 2.0
717 stars 66 forks source link

Backups that don't trigger any jobs never complete and aren't cleaned up #1005

Open haasad opened 3 months ago

haasad commented 3 months ago

Description

Backups that don't trigger any jobs are not marked as finished. Because they aren't marked as finished they are never cleaned up and accumulate endlessly on the cluster.

Additional Context

We automatically distribute a "default" k8up schedule to new namespaces on some of our clusters. If there's no PVCs (or BACKUP_SKIP_WITHOUT_ANNOTATION is true) and no other means of triggering a backup job (pods with k8up.io/backupcommand, prebackuppods etc.) the backup objects are created and never cleaned up. They accumulate endlessly.

Logs

No response

Expected Behavior

Backup objects that don't trigger any jobs should be marked as finished and cleaned up like regular finished backups.

Steps To Reproduce

  1. create a new empty namespace
  2. create a schedule:
    apiVersion: k8up.io/v1
    kind: Schedule
    metadata:
    name: noop-schedule
    spec:
    backup:
    failedJobsHistoryLimit: 2
    resources: {}
    schedule: "* * * * *"
    successfulJobsHistoryLimit: 2
  3. watch the backup objects accumulate:
    kubectl get backups.k8up.io --sort-by .metadata.creationTimestamp
    NAME                         SCHEDULE REF    COMPLETION   PREBACKUP              AGE
    noop-schedule-backup-pr7z4   noop-schedule                NoPreBackupPodsFound   27m
    noop-schedule-backup-4l9wt   noop-schedule                NoPreBackupPodsFound   26m
    noop-schedule-backup-262bq   noop-schedule                NoPreBackupPodsFound   25m
    <...>
    noop-schedule-backup-9f7b6   noop-schedule                NoPreBackupPodsFound   2m5s
    noop-schedule-backup-tvszx   noop-schedule                NoPreBackupPodsFound   65s
    noop-schedule-backup-8kkt4   noop-schedule                NoPreBackupPodsFound   5s

Version of K8up

v2.11.1

Version of Kubernetes

v1.28

Distribution of Kubernetes

OpenShift 4.15