Fix handling nuances of `src_mbuffer`

oetiker / znapzend

zfs backup with remote capabilities and mbuffer integration.

GNU General Public License v3.0

604 stars 136 forks source link

Follow up for PR #630 for issue #629

Example znapzendzetup list output with these fixes:

[2024-01-17 15:28:36.79858] [1624189] [info] WARNING: property 'src_mbuffer' not set on backup for rpool/ROOT, defaulting to 'off'
[2024-01-17 15:28:36.79925] [1624189] [info] WARNING: property 'src_mbuffer_size' not set on backup for rpool/ROOT, inheriting from legacy 'mbuffer_size': 128M
[2024-01-17 15:28:37.09402] [1624189] [info] WARNING: property 'dst_0_mbuffer' not set on backup for rpool/ROOT, inheriting path[:port] from legacy 'mbuffer': mbuffer
[2024-01-17 15:28:37.09426] [1624189] [info] WARNING: property 'dst_0_mbuffer_size' not set on backup for rpool/ROOT, inheriting from legacy 'mbuffer_size': 128M

...

*** backup plan: rpool/ROOT ***
           dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-deb/rpool/ROOT
   dst_0_mbuffer = mbuffer
dst_0_mbuffer_size = 128M
      dst_0_plan = 1month=>1week,1year=>1month,10years=>6months
         enabled = on
         mbuffer = mbuffer
    mbuffer_size = 128M
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = rpool/ROOT
     src_mbuffer = off
src_mbuffer_size = 128M
        src_plan = 1month=>1week,1year=>1month,10years=>6months
        tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ
      zend_delay = 0

Current properties in the dataset do not convey any mbuffer customizations (setup was created before these changes):

# zfs get -s local all rpool/ROOT | sort | grep znap
rpool/ROOT  org.znapzend:dst_0          znapzend:pond/export/DUMP/NUTCI/znapzend/ci-deb/rpool/ROOT  local
rpool/ROOT  org.znapzend:dst_0_plan     1months=>1weeks,1years=>1months,10years=>6months            local
rpool/ROOT  org.znapzend:enabled        on                                                          local
rpool/ROOT  org.znapzend:mbuffer        mbuffer                                                     local
rpool/ROOT  org.znapzend:mbuffer_size   128M                                                        local
rpool/ROOT  org.znapzend:post_znap_cmd  off                                                         local
rpool/ROOT  org.znapzend:pre_znap_cmd   off                                                         local
rpool/ROOT  org.znapzend:recursive      on                                                          local
rpool/ROOT  org.znapzend:src_plan       1months=>1weeks,1years=>1months,10years=>6months            local
rpool/ROOT  org.znapzend:tsformat       znapzend-auto-%Y-%m-%dT%H:%M:%SZ                            local
rpool/ROOT  org.znapzend:zend_delay     0                                                           local

On another system, there is no local mbuffer and we state that explicitly (as "off" in both legacy and SRC); related values like buffer sizes are successfully customized:

# znapzendzetup create --recursive --mbuffer=off --mbuffersize=256M --tsformat='znapzend-auto-%Y-%m-%dT%H:%M:%SZ' SRC '1month=>1week,1year=>1month,10years=>6months' zroot off 128M DST '1month=>1week,1year=>1month,10years=>6months' znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot off off mbuffer 1G
*** backup plan: zroot ***
           dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot
   dst_0_mbuffer = mbuffer
dst_0_mbuffer_size = 1G
      dst_0_plan = 1month=>1week,1year=>1month,10years=>6months
    dst_0_precmd = off
    dst_0_pstcmd = off
         enabled = on
         mbuffer = off
    mbuffer_size = 256M
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = zroot
     src_mbuffer = off
src_mbuffer_size = 128M
        src_plan = 1month=>1week,1year=>1month,10years=>6months
        tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ
      zend_delay = 0

Do you want to save this backup set [y/N]? y
NOTE: if you have modified your configuration, send a HUP signal
(pkill -HUP znapzend) to your znapzend daemon for it to notice the change.

# zfs get -s local all zroot | sort | grep znap
zroot  org.znapzend:dst_0               znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot  local
zroot  org.znapzend:dst_0_mbuffer       mbuffer                                                local
zroot  org.znapzend:dst_0_mbuffer_size  1G                                                     local
zroot  org.znapzend:dst_0_plan          1months=>1weeks,1years=>1months,10years=>6months       local
zroot  org.znapzend:dst_0_precmd        off                                                    local
zroot  org.znapzend:dst_0_pstcmd        off                                                    local
zroot  org.znapzend:enabled             on                                                     local
zroot  org.znapzend:mbuffer             off                                                    local
zroot  org.znapzend:mbuffer_size        256M                                                   local
zroot  org.znapzend:post_znap_cmd       off                                                    local
zroot  org.znapzend:pre_znap_cmd        off                                                    local
zroot  org.znapzend:recursive           on                                                     local
zroot  org.znapzend:src_mbuffer         off                                                    local
zroot  org.znapzend:src_mbuffer_size    128M                                                   local
zroot  org.znapzend:src_plan            1months=>1weeks,1years=>1months,10years=>6months       local
zroot  org.znapzend:tsformat            znapzend-auto-%Y-%m-%dT%H:%M:%SZ                       local
zroot  org.znapzend:zend_delay          0                                                      local

# znapzendzetup list
*** backup plan: zroot ***
           dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot
   dst_0_mbuffer = mbuffer
dst_0_mbuffer_size = 1G
      dst_0_plan = 1month=>1week,1year=>1month,10years=>6months
    dst_0_precmd = off
    dst_0_pstcmd = off
         enabled = on
         mbuffer = off
    mbuffer_size = 256M
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = zroot
     src_mbuffer = off
src_mbuffer_size = 128M
        src_plan = 1month=>1week,1year=>1month,10years=>6months
        tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ
      zend_delay = 0

# znapzendzetup create --recursive --tsformat='znapzend-auto-%Y-%m-%dT%H:%M:%SZ' SRC '1month=>1week,1year=>1month,10years=>6months' zroot DST '1month=>1week,1year=>1month,10years=>6months' znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot off off mbuffer 1G [2024-01-17 15:41:39.18773] [49409] [info] WARNING: property 'src_mbuffer' not set on backup for zroot, inheriting 'off' from legacy 'mbuffer' [2024-01-17 15:41:39.18809] [49409] [info] WARNING: property 'src_mbuffer_size' not set on backup for zroot, inheriting from legacy 'mbuffer_size': 1G *** backup plan: zroot *** dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot dst_0_mbuffer = mbuffer dst_0_mbuffer_size = 1G dst_0_plan = 1month=>1week,1year=>1month,10years=>6months dst_0_precmd = off dst_0_pstcmd = off enabled = on mbuffer = off mbuffer_size = 1G post_znap_cmd = off pre_znap_cmd = off recursive = on src = zroot src_mbuffer = off src_mbuffer_size = 1G src_plan = 1month=>1week,1year=>1month,10years=>6months tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ zend_delay = 0

zfs get -s local all zroot | sort | grep znap

zroot org.znapzend:dst_0 znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot local zroot org.znapzend:dst_0_mbuffer mbuffer local zroot org.znapzend:dst_0_mbuffer_size 1G local zroot org.znapzend:dst_0_plan 1months=>1weeks,1years=>1months,10years=>6months local zroot org.znapzend:dst_0_precmd off local zroot org.znapzend:dst_0_pstcmd off local zroot org.znapzend:enabled on local zroot org.znapzend:mbuffer off local zroot org.znapzend:mbuffer_size 1G local zroot org.znapzend:post_znap_cmd off local zroot org.znapzend:pre_znap_cmd off local zroot org.znapzend:recursive on local zroot org.znapzend:src_mbuffer off local zroot org.znapzend:src_mbuffer_size 1G local zroot org.znapzend:src_plan 1months=>1weeks,1years=>1months,10years=>6months local zroot org.znapzend:tsformat znapzend-auto-%Y-%m-%dT%H:%M:%SZ local zroot org.znapzend:zend_delay 0 local

* legacy-style setup, with `--mbuffer=mbuffer` (no port part) and no mention in SRC -- `src_mbuffer` is decided as `off` per legacy default for non-networked mode:

znapzendzetup create --recursive --mbuffer=mbuffer --mbuffersize=256M --tsformat='znapzend-auto-%Y-%m-%dT%H:%M:%SZ' SRC '1month=>1week,1year=>1month,10years=>6months' zroot DST '1month=>1week,1year=>1month,10years=>6months' znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot off off mbuffer 1G

[2024-01-17 15:44:06.55401] [49469] [info] WARNING: property 'src_mbuffer' not set on backup for zroot, defaulting to 'off' [2024-01-17 15:44:06.55438] [49469] [info] WARNING: property 'src_mbuffer_size' not set on backup for zroot, inheriting from legacy 'mbuffer_size': 256M backup plan: zroot dst_0 = znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot dst_0_mbuffer = mbuffer dst_0_mbuffer_size = 1G dst_0_plan = 1month=>1week,1year=>1month,10years=>6months dst_0_precmd = off dst_0_pstcmd = off enabled = on mbuffer = mbuffer mbuffer_size = 256M post_znap_cmd = off pre_znap_cmd = off recursive = on src = zroot src_mbuffer = off src_mbuffer_size = 256M src_plan = 1month=>1week,1year=>1month,10years=>6months tsformat = znapzend-auto-%Y-%m-%dT%H:%M:%SZ zend_delay = 0

Do you want to save this backup set [y/N]? ...

zfs get -s local all zroot | sort | grep znap

zroot org.znapzend:dst_0 znapzend:pond/export/DUMP/NUTCI/znapzend/ci-bsd/zroot local zroot org.znapzend:dst_0_mbuffer mbuffer local zroot org.znapzend:dst_0_mbuffer_size 1G local zroot org.znapzend:dst_0_plan 1months=>1weeks,1years=>1months,10years=>6months local zroot org.znapzend:dst_0_precmd off local zroot org.znapzend:dst_0_pstcmd off local zroot org.znapzend:enabled on local zroot org.znapzend:mbuffer mbuffer local zroot org.znapzend:mbuffer_size 256M local zroot org.znapzend:post_znap_cmd off local zroot org.znapzend:pre_znap_cmd off local zroot org.znapzend:recursive on local zroot org.znapzend:src_mbuffer off local zroot org.znapzend:src_mbuffer_size 256M local zroot org.znapzend:src_plan 1months=>1weeks,1years=>1months,10years=>6months local zroot org.znapzend:tsformat znapzend-auto-%Y-%m-%dT%H:%M:%SZ local zroot org.znapzend:zend_delay 0 local

oetiker / znapzend