search

wamdam / backy2

backy2: Deduplicating block based backup software for ceph/rbd, image files and devices
http://backy2.com/
Other
196 stars 39 forks source link

A way to do remote backup #7

Closed dyasny closed 2 years ago

dyasny commented 7 years ago

There is no way to have a backy2 agent working on a backed-up machine and pushing backups back to the backup server.

Currently, I have a server dedicated to backy2, with lots of backup space and the sqlite DB in place. If that server can reach the backed up file or block device, it works great.

In my (quite common) scenario, I need to back up VMs located on a SAN based LVM store. The backup server is a network infrastructure machine and has no HBA to access the SAN directly. The only hosts that can access the SAN and hypervisors with minimal disk space available. If I install backy2 on a hypervisor, I have to somehow make it reach the storage space on the backup server and the sqlite database.

I would love to be able to install a backy2 daemon on a hypervisor, and let it send over the bits it can see from the SAN over to the backup server directly. Alternatively, I could probably export the backy2 server's storage space over NFS to one of the hypervisors, but I'm not so sure about the database.

wamdam commented 7 years ago

This would just need two things:

  1. a back2 io module which implements https://github.com/wamdam/backy2/blob/master/src/backy2/io/__init__.py and uses the network to get information
  2. a standalone daemon which provides these methods over the network

Relatively simple thing. I'd say sponsoring 2 days should make this possible. Contact me if you're interested. Also, if you like, you can send patches ;)

dyasny commented 7 years ago

Thanks for the reply. I'm not sure I have the time to send in patches, but I can probably do an ugly workaround for this one (iscsi or aoe export or backy2 in a VM with the LUN passed through)

errieman commented 5 years ago

Has there been made any progress on this? I'm in the same kind of situation, but with Ceph. I'd like to back up a Ceph cluster to an offsite backup cluster. The requirements to make this possible would be somewhat similar to the requirements for the situation @dyasny is/was in, on the network side at least.

wamdam commented 5 years ago

We do exactly this (in case you mean rbd backups) with backy2 to a remote s3 store ootb.

errieman commented 5 years ago

My initial idea was to run backy2 on the backup cluster itself so that it would pull the images, because there are multiple live clusters in multiple remote locations. To use the s3 backend would require backy2 to run on every cluster itself, which also is a viable solution. I overlooked that one. And now I think of it, I guess that is how backy2 is made to use.

Thank you for your reply.

wamdam commented 5 years ago

You're very welcome. Actually that's also the most performant way (and that's actually why backy was designed that way) because backy will only send data blocks which have changed to the data backend. You may even run simple single postgresql databases on your locations and use "backy2 export" to another s3 store (we use s3cmd for this) to also backup metadata.

wamdam commented 2 years ago

This has not been sponsored until now and actually we have an internal solution now, so I'm not interested in performing that work with backy2. I'll close this request.