Runonce with manually provided snapname (--forcedSnapshotSuffix=X)

[jimklimov]

Somewhat a follow-up on discussion from #455 - if sysadmins intend to make a specially-named snapshot for their unscheduled backup, let znapzend help them do it more easily.

CC @lotheac and @oetiker

[jimklimov]

Note: so far the idea was coded only, not tested IRL :)

In particular, not sure initially if this snapshot name that does not match the patterns configured in the backup plan would actually get picked up for sending, more so if/after #455 is merged to jump over some snaps and do fewer incremental operations.

[coveralls]

Coverage increased (+0.05%) to 91.609% when pulling b0c9b92d9875ce24c8283bdc8fef38f15ab0c0aa on jimklimov:runonce-snapname into 1a55c7331cf29a549fd154572a452c9dd1dacd29 on oetiker:master.

[jimklimov]

The extended test was not in vain - our t/zfs mock in fact does not get properties for all datasets as was expected of it.

[jimklimov]

And we have green results with not ok tests in the middle :\

[oetiker]

tere is a conflict it seems

[jimklimov]

Split out the commits for arg-less runonce into another PR (#458) leaving only the work on forced snapname in this branch.

[jimklimov]

Marking as "WIP" to figure out the test fails...

[jimklimov]

Back to drawing board after some local tests. Seems this needs two filters, for sends and for cleanups (to not kill manually made snapname). And with manually specified snapnames, maybe needs a pattern variable option too...

[jimklimov]

Houston, we have a... subject for discussion.

As I experimented with this code in a practical system, I got it to work for passing both just-created "forced" snapshot names as well as automatically generated ones, but only if the "last common" snapshot on dest and src exists and is newer than anything not included by the timestamp filter. Otherwise, zfs recv suggests that offending snapshots on destination should be destroyed first and so denies the replication.

I see a few ways out of this:

• As in the recent PR codebase state, effectively look at all snapshots when searching for a last-common snapshot, allowing to replicate starting from both automatic and forced names (and yet look at only timestamp-patterns for cleaning); ** Maybe fall back to this wider filter if the original replication attempt with timestamp-based filter fails with specific conditions (no common snaps found, e.g. when the forced one is the first and only made or when not synced for too long; when receive failed because of conflict needing removal of newer snaps on destination)?
• When forcing a name, also create a next snapshot by timestamp generated pattern, and make sure both of them are replicated... this is the discussion point: I am not sure how well this would work WRT policy cleaning of older snaps - is it possible that it would remove latest timestamp-patterned snapshots in a way that an older "forced-named" one becomes de-facto latest but not visible to the timestamp-filtering send and clean operations? ** Note especially with the skipIntermediates feature, to make sure "forced-named" snaps are replicated, whether initially or on a re-run (if initial forced-replication broke before completion, so at least one of the latest two or so snapshots is not a timestamp-based pattern);
• on presumably all freely or not available zfs versions (tested in current OI and on Sun Solaris 10u8), we can actually set custom "local" attributes into a snapshot... these could be used to identify special datasets that are not in timestamp pattern, for use as lastCommon and as required for replicating.

[oetiker]

can you state the usecase for this feature, then it might be easier to figure the right way of implementing it ... my first impression is that we should avoid trying to make znapzend behavior more complex to understand than it already is.

afaik the latest date-time stapshot will never get cleaned

[jimklimov]

Sorry for falling out of the loop a bit. I think original post still holds here:

if sysadmins intend to make a specially-named snapshot for their unscheduled backup, let znapzend help them do it more easily.