search

Lirt / velero-plugin-for-openstack

Openstack Cinder, Manila and Swift plugin for Velero backups
MIT License
26 stars 16 forks source link

Incremental pv backup #61

Closed AlwaySummit closed 1 year ago

AlwaySummit commented 1 year ago

Hi @Lirt Have to bug you again. :) I'm just wondering if plugin supports incremental backup for snapshot? I know restic might be an alternative but it's kind of a big change for us. Would you share some insights? Thanks ahead.

Lirt commented 1 year ago

Hi @AlwaySummit,

Velero plugin interface for volume backups was written with snapshots and volumes in mind and is not doing traditional backups as you might think of and therefore there is no term like incremental backup.

Effectively the backup of PV in this plugin is volume created from a cinder snapshot of the volume behind PVC. This snapshot sometimes cannot be considered as a proper backup because it's not stored off-site and will not survive serious datacenter incidents.

There is one comment I left about volume backups in readme

Snapshots in the cinder backend are not always supposed to be used as durable. In some cases for proper availability, the snapshot need to be backed up to off-site storage. Please consult if your cinder backend creates durable snapshots with your cloud provider.

Cinder API supports also traditional backups including incremental feature (https://docs.openstack.org/cinder/latest/admin/volume-backups.html). This however depends on Openstack installation and may not fit to the plugin interface. For example for previous two Openstack private clouds I worked with this feature was not configured.

If you think this feature is supported in your Openstack cloud, you can use it instead of this plugin or you can keep this plugin creating snapshot backups and then use openstack volume backup create to backup them to Swift.

Probably best way to create off-site backups with Velero is using Restic which according to my knowledge supports incremental backups.

Let me know if this is what you were asking about. I tried to give you complete explanation as this topic is fairly complicated.

AlwaySummit commented 1 year ago

@Lirt Thanks for the insights. That would be very helpful. Does it mean that incremental backup for pv is just platform independent and velero doesn't intend to support such feature? The only two options are using restic or using platform(openstack) functionalities to create backup.

Lirt commented 1 year ago

I mean it's not possible to create incremental volume/disk in cloud technically. And backups done using Velero VolumeSnapshot provider are just full copies of standard disks (volumes in Openstack terminology).

Restic is uploading data as objects into object storage. So this way it can calculate difference between what is going to upload and what is uploaded already and therefore make incremental backup.

If you need more durability, you need to use restic and store the backup off-site.

Additional notes: Let's say for GCP, if your backup disk is regional it can be durable enough for you, because GCP will guarantee that unless both 2 zones where disk is replicated burns out, your data is safe.

If your openstack cloud provider guarantees that your disk is backed by replicated storage in multiple zones, it could be sufficient for backup.

Everything depends on your backup or DR strategy and requirements.

So standard way this plugin does backup can be enough for one person and not enough for other and depends on storage implementation of specific Openstack cloud.

AlwaySummit commented 1 year ago

Thanks @Lirt. We'll turn back to Restic then.