search

oetiker / znapzend

zfs backup with remote capabilities and mbuffer integration.
www.znapzend.org
GNU General Public License v3.0
608 stars 138 forks source link

dst_plan = 1second=>1second #413

Closed Harvie closed 5 years ago

Harvie commented 5 years ago

I want to keep destination synced as often as possible (but no need to preserve all these frequent snapshots). Can i use following dst_plan?

1second=>1second,7days=>1hour

Harvie commented 5 years ago

OK. I've tried and it seems to be sub-optimal. It just keeps making lots of snapshots instead of syncing them...

# zfs snapshot -r tank@2019-03-27-153513
# zfs list -H -r -o name -t filesystem,volume tank
# zfs list -H -o name -t snapshot -s creation -d 1 tank
# ssh -o batchMode=yes -o ConnectTimeout=30 root@host zfs list -H -o name -t snapshot -s creation -d 1 tank/backup/virt1
# zfs snapshot -r tank@2019-03-27-153514
# zfs snapshot -r tank@2019-03-27-153515
# zfs snapshot -r tank@2019-03-27-153516
# zfs snapshot -r tank@2019-03-27-153517
# zfs snapshot -r tank@2019-03-27-153518
# zfs snapshot -r tank@2019-03-27-153519
# zfs snapshot -r tank@2019-03-27-153520
# zfs snapshot -r tank@2019-03-27-153521
# zfs snapshot -r tank@2019-03-27-153522
# zfs snapshot -r tank@2019-03-27-153523
# zfs snapshot -r tank@2019-03-27-153524
# zfs snapshot -r tank@2019-03-27-153525
# zfs snapshot -r tank@2019-03-27-153526
# zfs snapshot -r tank@2019-03-27-153527
# zfs snapshot -r tank@2019-03-27-153528
# zfs send -I tank@2019-03-27-150000 tank@2019-03-27-153513|ssh -o batchMode=yes -o ConnectTimeout=30 'root@host' 'zfs recv -F tank/backup/virt1'
# zfs snapshot -r tank@2019-03-27-153529
# zfs snapshot -r tank@2019-03-27-153530
# zfs snapshot -r tank@2019-03-27-153531
# zfs snapshot -r tank@2019-03-27-153532
# zfs snapshot -r tank@2019-03-27-153533
# zfs snapshot -r tank@2019-03-27-153534
# zfs snapshot -r tank@2019-03-27-153535
# zfs snapshot -r tank@2019-03-27-153536
# zfs snapshot -r tank@2019-03-27-153537
# zfs snapshot -r tank@2019-03-27-153538
# zfs snapshot -r tank@2019-03-27-153539
# zfs snapshot -r tank@2019-03-27-153540
# zfs snapshot -r tank@2019-03-27-153541
# zfs snapshot -r tank@2019-03-27-153542
# zfs snapshot -r tank@2019-03-27-153543
# zfs snapshot -r tank@2019-03-27-153544
# zfs list -H -o name -t snapshot -s creation -d 1 tank/backup
# ssh -o batchMode=yes -o ConnectTimeout=30 root@host zfs list -H -o name -t snapshot -s creation -d 1 tank/backup/virt1/backup
# zfs snapshot -r tank@2019-03-27-153545
# zfs snapshot -r tank@2019-03-27-153546
# zfs snapshot -r tank@2019-03-27-153547
# zfs snapshot -r tank@2019-03-27-153548
# zfs snapshot -r tank@2019-03-27-153549
# zfs snapshot -r tank@2019-03-27-153550
# zfs snapshot -r tank@2019-03-27-153551
# zfs snapshot -r tank@2019-03-27-153552
# zfs snapshot -r tank@2019-03-27-153553
# zfs snapshot -r tank@2019-03-27-153554
# zfs snapshot -r tank@2019-03-27-153555
# zfs snapshot -r tank@2019-03-27-153556
# zfs snapshot -r tank@2019-03-27-153557
# zfs snapshot -r tank@2019-03-27-153558
# zfs snapshot -r tank@2019-03-27-153559
# zfs snapshot -r tank@2019-03-27-153600
# zfs send -I tank/backup@2019-03-27-150000 tank/backup@2019-03-27-153544|ssh -o batchMode=yes -o ConnectTimeout=30 'root@host' 'zfs recv -F tank/backup/virt1/backup'
# zfs snapshot -r tank@2019-03-27-153601
# zfs snapshot -r tank@2019-03-27-153602
# zfs snapshot -r tank@2019-03-27-153603
# zfs snapshot -r tank@2019-03-27-153604
# zfs snapshot -r tank@2019-03-27-153605
# zfs snapshot -r tank@2019-03-27-153606
# zfs snapshot -r tank@2019-03-27-153607
oetiker commented 5 years ago

this is because it only starts a new send once the previous one has completed .... so you maybe should cut down on the frequency a little

and also keeping the 1 second snapshots for 1 second only might not be ideal also since there must besome overlap for the syncing to work

Harvie commented 5 years ago

Turned out i had enabled iptables ratelimit on ssh port, which made performace much worse, because znapzend makes a lots of ssh connections. Now that i've disabled the ratelimit i am able to sync within few seconds... So far i am happy with how znapzend works :-)