search

jgoerzen / simplesnap

A simple and powerful way to send ZFS snapshots across a network
GNU General Public License v3.0
49 stars 15 forks source link

Potential for removing "-F" on receive #4

Open treydock opened 9 years ago

treydock commented 9 years ago

I've noticed that one side-effect of using "-F" for a receive is that all the snapshots that exist on the archival host are removed if they were also removed on the host being backed up. This presents an issue if the goal of an archival host is to retain snapshots longer than the system its backing up. My initial idea for archival was to keep hourly for a day, daily for a week, and weekly for a month on a production system and double those periods on the archival host.

What would be, or have been, the issues of not using the "-F" flag?

Thanks,

notr1ch commented 7 years ago

I was looking into this as my requirements are similar to what you want - lots of snapshots on the backup host, not as many on the target due to space constraints. I'm very new to ZFS, but from what I can tell, -F is needed to rollback the dataset to the previous snapshot so that the incremental send can function, as without -F there is a risk that the dataset has been modified on the backuphost and thus the incremental sends won't be in sync (I hope this is is explained right!). Another effect of -F is that it deletes any snapshots not present on the sender (why?!).

Using zfs rollback seems like it could accomplish the same goal to avoid a modified dataset. For now I've removed -F from simplesnap and made the target datasets read-only on the backuphost so there's no risk of accidental modification. This seems to work so far.. fingers crossed.

Update: It seems the readonly attribute is lost after the zfs receive so this isn't an ideal solution.

notr1ch commented 7 years ago

After putting this into production, it's failing with errors like "cannot receive incremental stream: destination tank/backups/db has been modified" or "cannot receive incremental stream: invalid backup stream". Unfortunately without the ability to maintain snapshots on the backup host, simplesnap isn't going to work for me.

jgoerzen commented 7 years ago

Is that with -F removed?

I've been using simplesnap (as written) for a long time now and have never seen a problem like this.

notr1ch commented 7 years ago

Sorry I should have been more explicit, that was with -F removed. I did notice that due to a long run time, a snapshot on the backup host was created by a cron job in the middle of a receive operation, which I assume was responsible for one of those errors.

I've wiped everything out and started from scratch, if I continue to experience issues I'll go back to using -F and try to come up with some other way to achieve what I want (cloning?)

jgoerzen commented 7 years ago

It is not an issue to create snapshots on the machine that is sending the backups while a backup is running. It is probably unwise to be creating new snapshots over the backup data on the system that receives the backups. They will necessarily have to be removed anyhow.

On 03/24/2017 09:44 AM, Richard Stanway wrote:

Sorry I should have been more explicit, that was with -F removed. I did notice that due to a long run time, a snapshot on the backup host was created by a cron job in the middle of a receive operation, which I assume was responsible for one of those errors.

I've wiped everything out and started from scratch, if I continue to experience issues I'll go back to using -F and try to come up with some other way to achieve what I want (cloning?)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jgoerzen/simplesnap/issues/4#issuecomment-289076920, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG5HR3DqXCpQTnyv33srYoSAKYKnZWPks5ro_LzgaJpZM4E8Tq6.

notr1ch commented 7 years ago

As long as the automatically created simplesnap snapshot is the latest snapshot for the dataset, shouldn't it be OK to maintain older snapshots of the backup data? While admittedly I'm very new to ZFS, I don't really understand why it's necessary to remove the snapshots on the backup dataset to receive the latest one.

jplitza commented 6 years ago

-F doesn't remove snapshots on the receiving host that are older than the anchor snapshot, i.e. the newest snapshot by simplesnap. It does a rollback of the receiving side to the anchor snapshot, which IIRC is necessary because simplesnap creates lockfiles inside the datasets themselves. But snapshots older than the anchor snapshot that were manually created (or at least not created by simplesnap) are never touched by simplesnap.