search

NetApp / trident

Storage orchestrator for containers
Apache License 2.0
762 stars 222 forks source link

CommVault Backup / Trident - How to handle "RETAIN"-PVC's #942

Open chrs04 opened 3 weeks ago

chrs04 commented 3 weeks ago

Hi,

we are using CommVault for our Openshift Backup and ive observed the following behavior with PVC's that have the ReclaimPolicy "RETAIN":

  1. Backup starts and PVC's are getting gecloned
  2. PV's are getting cloned on the netapp
  3. Backup are performed
  4. PVs are split (split clone)
  5. PVs with ReclaimPolicy DELETE are removed from netapp.
  6. PVs with ReclaimPolicy RETAIN are retained

How do you handle this with your backup solution? We also observed this behavior with a manual "Clone" out of openshift. Is there some kind of setting to change this behavior?

I also raised a case at CommVault.

ntap-johanneswagner commented 3 weeks ago

Hi chrs04,

you are facing standard k8s behavior: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reclaiming

As long as the policy is set to retain, the volumes will not be deleted, even if the pv is deleted. The correct method to clean up everything would be to patch the reclaim policy of the pv from retain to delete. With this, trident should recognize that the PV isn't in use anymore and clean up everything including the storage. This is something that should be scheduled by CommVault as soon as the backup expires.

chrs04 commented 3 weeks ago

Thank you @ntap-johanneswagner - Thats also a good approach to patch the reclaim policy after the retention is met. Another would be to change it right after the backup is completed.

Do you know how other backup tools like Kasten (Veeam) handle those situations?

ntap-johanneswagner commented 3 weeks ago

@chrs04 it really depends on the workflow whether you want to patch it after the backup or after the retention is expired. In both cases, the Backupvendor should know what works best for them and their architecture. I don't know how casten behaves there, I know that you could configure the behavior on your own for all velero based tools like OADP, Dell PowerProtect etc.