backube / volsync

Asynchronous data replication for Kubernetes volumes
https://volsync.readthedocs.io
GNU Affero General Public License v3.0
570 stars 67 forks source link

Add rdiff-backup replication method #1029

Open kyrofa opened 10 months ago

kyrofa commented 10 months ago

Describe the feature you'd like to have. The ability to replicate via rdiff-backup. This is our current backup method, but as we move more products to k8s, it has gotten harder to pull off. We started investigating writing an operator to take a snapshot and use rdiff-backup (or perhaps use SnapScheduler in combination with a more simple cron job), but then we discovered VolSync, which is almost a perfect fit except for this lack of support. Instead of building our own, I'd be happy to contribute this support to VolSync, but I wanted to first log an issue and gauge interest.

What is the value to the end user? (why is it a priority?) rdiff-backup is a reverse incremental backup tool that uses the rsync protocol. It's not really suitable for replication, to be clear, but a way to take space-efficient backups. It generally works over SSH.

How will we know we have a good solution? (acceptance criteria) A new replication method that looks a bit like the current rsync SSH method, something like the following:

---
apiVersion: volsync.backube/v1alpha1
kind: ReplicationSource
# snip...
spec:
  # snip...
  rdiff-backup:
    sshKeys: volsync-rdiff-dest-sshkeys
    address: my.host.com
    copyMethod: Clone
tesshuflower commented 10 months ago

@kyrofa I've discussed with @JohnStrunk and think we would potentially be interested in this if you're willing to contribute and help maintain the mover. Copying some comments John had from a request for a kopia mover:

By the end of the process, we need to have:

Since this will be a fairly substantial piece, starting w/ a design PR is probably a good place to begin.

We'd have a few questions at the start:

Neustradamus commented 10 months ago

To follow this ticket