search

backube / snapscheduler

Scheduled snapshots for Kubernetes persistent volumes
https://backube.github.io/snapscheduler/
GNU Affero General Public License v3.0
262 stars 26 forks source link

Actively used VolumeSnapshots being deleted due to retention violation #667

Open Chase-Marino opened 3 months ago

Chase-Marino commented 3 months ago

Describe the bug When restoring to a VolumeSnapshot created by SnapshotScheduler which has retention defined, you are subject to your VolumeSnapshot being deleted. This results in the PVC having a dangling reference.

Steps to reproduce

  1. Create a snapshot schedule with retention time/count defined.
  2. Create PVC referencing VolumeSnapshot created by schedule
  3. Wait for retention cycle
  4. Observe the actively used VolumeSnapshot is deleted

Expected behavior VolumeSnapshot should be protected/excluded from retention once it has been referenced.

Actual results VolumeSnapshot is deleted

Additional context Current workaround is to delete snapscheduler.backube/when, snapscheduler.backube/schedule labels.

Solution ideas are for operator to detect when it was mounted and permanently exclude that Snapshot, or for a snapscheduler.backube/retention="keep" label to exclude a Snapshot from retention cleanup

Chase-Marino commented 3 months ago

@JohnStrunk Thoughts?

JohnStrunk commented 3 months ago

I'm a bit confused about what you believe the problem to be with the current behavior.

When the snapshot name is used in the dataSource field for a PVC, it serves to set the initial contents of the PVC, but once that PVC has been provisioned, the value of the field is meaningless. Why do you believe the associated snapshot should be retained?

BTW, removing snapscheduler.backube/schedule is the correct approach if you want to retain the snapshot.