oetiker / znapzend

zfs backup with remote capabilities and mbuffer integration.
www.znapzend.org
GNU General Public License v3.0
604 stars 136 forks source link

Handle `enabled=off` with recursion in place #625

Closed jimklimov closed 3 months ago

jimklimov commented 5 months ago

(Includes and) Follows up from #623 with a practical feature: if the same intermediate dataset that declares enabled=off also says recursive=on, the disablement takes place for descendants as well.

As discussed among ideas in #623, this PR adds recognition for datasets (descendants of a "backupSet") which declare not only an org.znapzend:enabled value but optionally also an org.znapzend:recursive one (so only these one or two ZFS properties are defined in a local source dataset under the one which defines a full retention schedule).

It also somewhat reasonably defines behavior for such dataset trees where a backupSet with a full retention schedule has some descendant which defines an enabled=off behavior (individual or recursive), and some of its descendants re-enable the backup (individually or recursively). I'm sure this can be tested with mocks, but out of time at least today, maybe longer, to pursue testing. Worked as expected in a real-life setup (which however did not try all edge cases) :)

This is also seen in znapzendzetup list outputs:

...
*** backup plan: rpool/home ***
           dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-deb/rpool/home
      dst_0_plan = 1week=>1day,1month=>1week,1year=>1month,10years=>6months
         enabled = on
         mbuffer = off
    mbuffer_size = 128M
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = rpool/home
        src_plan = 1week=>1day,1month=>1week,1year=>1month,10years=>6months
        tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ
      zend_delay = 0

*** backup plan: rpool/home/abuild/.ccache ***
         enabled = off
             src = rpool/home/abuild/.ccache

*** backup plan: rpool/home/abuild/jenkins-nut-altroots ***
         enabled = off
       recursive = on
             src = rpool/home/abuild/jenkins-nut-altroots

...
github-actions[bot] commented 5 months ago

@check-spelling-bot Report jimklimov/znapzend: enabled-off-with-recursion into -> oetiker/znapzend: 100c9f629894777fba32055422304e5e55fed367

:red_circle: Please review

See the :open_file_folder: files view, the :scroll:action log, or :memo: job summary for details.

Unrecognized words (5)

hashmaps idx LRds rindex substr

Previously acknowledged words that are now absent aix Balert Bcreate Bdebug Bdelete Bedit Berr Bexport Bimport Binfo Bnoaction Bnot Bpidfile Bpost Bpre Bsyslog Bwarning Bzfs Bznapzend Bznapzendzetup Bznapzendztatz CBuilder cpanfile cpanm cpanmin CPANSNAPV crt Cwd cygwin DBD DESTDIR distdir DTDs endif EXTRADIST Fcntl forkcall Icommand Icommon Icreate Idataset Idestroy Idocuments Iexport Ifacility Ifeature Ifilepath Ihome Ilimited imandir Inumber Ioptions Ipath Ipictures Irecursive Isend Iskip Isnapshots Isnapsuffix Isources Itank Ithirdparty Itimeout Iusbbackup Iuser Ivalue Iznapzendzetup JBERGER LEONT lpr Mkbootstrap nobase notest nroff ODBC Pipely RCAPUTO SUBDIRS svcdir troff unicode utf VOS vroff xargs 🫥
To accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands ... in a clone of the [https://github.com/oetiker/znapzend](https://github.com/oetiker/znapzend) repository on the `master` branch ([:information_source: how do I use this?]( https://github.com/check-spelling/check-spelling/wiki/Accepting-Suggestions)): ``` sh curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/v0.0.22/apply.pl' | perl - 'https://github.com/oetiker/znapzend/actions/runs/7533184333/attempts/1' ```
Available :books: dictionaries could cover words (expected and unrecognized) not in the :blue_book: dictionary This includes both **expected items** (649) from .github/workflows//spelling/expect.txt and **unrecognized words** (5) Dictionary | Entries | Covers | Uniquely -|-|-|- [cspell:software-terms/dict/softwareTerms.txt](https://raw.githubusercontent.com/check-spelling/cspell-dicts/v20230509/dictionaries/software-terms/dict/softwareTerms.txt)|1288|82|18| [cspell:php/dict/php.txt](https://raw.githubusercontent.com/check-spelling/cspell-dicts/v20230509/dictionaries/php/dict/php.txt)|1689|59|6| [cspell:python/src/python/python-lib.txt](https://raw.githubusercontent.com/check-spelling/cspell-dicts/v20230509/dictionaries/python/src/python/python-lib.txt)|2417|54|6| [cspell:node/dict/node.txt](https://raw.githubusercontent.com/check-spelling/cspell-dicts/v20230509/dictionaries/node/dict/node.txt)|891|56|5| [cspell:filetypes/filetypes.txt](https://raw.githubusercontent.com/check-spelling/cspell-dicts/v20230509/dictionaries/filetypes/filetypes.txt)|264|19|4| Consider adding them (in `.github/workflows/spelling.yml`): ``` yml with: extra_dictionaries: cspell:software-terms/dict/softwareTerms.txt cspell:php/dict/php.txt cspell:python/src/python/python-lib.txt cspell:node/dict/node.txt cspell:filetypes/filetypes.txt ``` To stop checking additional dictionaries, add (in `.github/workflows/spelling.yml`): ``` yml check_extra_dictionaries: '' ```
jimklimov commented 5 months ago

Thanks for the review, and the master-branch bump (docker fix), nice to see it all claim to be green again :)

removing the extra snapshots quickly seems like a good move!

I don't think this PR tackles it directly - this is part of createSnapshot() and was for a while (since #154 at least?); this change here (and started by "externalization" of code in #623) modifies how users can state more easily which datasets are not-enabled for znapzend handling so their snapshots would be removed earlier. Also #626 takes care of skipping zfs send for whatever datasets are in the not-enabled lists (also builds upon #623, but is independent of the change in this PR -- so if that one is merged alone, it would be about skipping exactly the datasets with enabled=off and no care for recursions).

jimklimov commented 5 months ago

Hm, I don't actually see any docs about the enabled=off handling :\

UPDATE: Actually, README.md starts out to say how this is not supported :) I'll claim it as "experimental" instead and document as such.

jimklimov commented 4 months ago

Ok, so after the merge of groundwork from #623 the specific changes of this PR are much better visible :)