search

jimsalterjrs / sanoid

These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
http://www.openoid.net/products/
GNU General Public License v3.0
3.14k stars 308 forks source link

Sanoid Sync question #889

Closed madmax01 closed 8 months ago

madmax01 commented 9 months ago

Hi team,

i cannot get the syncoid working.. init works then next day it blames

related to this https://github.com/jimsalterjrs/sanoid/issues/878

Replication HOST [pool1] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_system] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_files] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_images] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/backup] recursive = yes frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autoprune = yes autosnap = no

SOURCE HOST:

[pool1] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_system] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_images] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

[pool1/local_files] frequently = 0 hourly = 0 daily = 14 monthly = 0 yearly = 0 autosnap = yes autoprune = yes

ZFS LIST on Replica host

NAME USED AVAIL REFER MOUNTPOINT pool1 10.6G 109T 104K /zfs/pool1 pool1/backup 2.09M 109T 120K /zfs/pool1/backup pool1/backup/onehost-ty-0009 508K 109T 144K /zfs/pool1/backup/onehost-ty-0009 pool1/backup/onehost-ty-0009/local_files 96K 109T 96K /zfs/pool1/backup/onehost-ty-0009/local_files pool1/backup/onehost-ty-0009/local_images 96K 109T 96K /zfs/pool1/backup/onehost-ty-0009/local_images pool1/backup/onehost-ty-0009/local_system 108K 109T 108K /zfs/pool1/backup/onehost-ty-0009/local_system pool1/backup/onehost-ty-0010 500K 109T 136K /zfs/pool1/backup/onehost-ty-0010 pool1/backup/onehost-ty-0010/local_files 96K 109T 96K /zfs/pool1/backup/onehost-ty-0010/local_files pool1/backup/onehost-ty-0010/local_images 96K 109T 96K /zfs/pool1/backup/onehost-ty-0010/local_images pool1/backup/onehost-ty-0010/local_system 108K 109T 108K /zfs/pool1/backup/onehost-ty-0010/local_system pool1/backup/onehost-ty-0011 508K 109T 136K /zfs/pool1/backup/onehost-ty-0011 pool1/backup/onehost-ty-0011/local_files 96K 109T 96K /zfs/pool1/backup/onehost-ty-0011/local_files pool1/backup/onehost-ty-0011/local_images 96K 109T 96K /zfs/pool1/backup/onehost-ty-0011/local_images pool1/backup/onehost-ty-0011/local_system 108K 109T 108K /zfs/pool1/backup/onehost-ty-0011/local_system pool1/backup/onehost-ty-0012 508K 109T 144K /zfs/pool1/backup/onehost-ty-0012 pool1/backup/onehost-ty-0012/local_files 96K 109T 96K /zfs/pool1/backup/onehost-ty-0012/local_files pool1/backup/onehost-ty-0012/local_images 96K 109T 96K /zfs/pool1/backup/onehost-ty-0012/local_images pool1/backup/onehost-ty-0012/local_system 108K 109T 108K /zfs/pool1/backup/onehost-ty-0012/local_system pool1/local_files 96K 109T 96K /var/lib/one/datastores/2 pool1/local_images 10.5G 109T 10.5G /var/lib/one/datastores/1 pool1/local_system 96K 109T 96K /var/lib/one/datastores/0

init it blamed "pool1/backup" not existing.. i created it manual

then i synced and it AUTO created all the entries... init sync cool

then when 2nd day sync kicks into it blames this

sanoid --monitor-snapshots CRIT: pool1/backup has no daily snapshots at all!, CRIT: pool1/backup/onehost-ty-0009 has no daily snapshots at all!, CRIT: pool1/backup/onehost-ty-0009/local_files has no daily snapshots at all!, CRIT: pool1/backup/onehost-ty-0009/local_images has no daily snapshots at all!, CRIT: pool1/backup/onehost-ty-0009/local_system has no daily snapshots at all!, CRIT: pool1/backup/onehost-ty-0010 newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0010/local_files newest daily snapshot is 2d 9h 26m 12s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0010/local_images newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0010/local_system newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0011 newest daily snapshot is 2d 9h 26m 12s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0011/local_files newest daily snapshot is 2d 9h 26m 12s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0011/local_images newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0011/local_system newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0012 newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0012/local_files newest daily snapshot is 2d 9h 26m 12s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0012/local_images newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s), CRIT: pool1/backup/onehost-ty-0012/local_system newest daily snapshot is 2d 9h 26m 11s old (should be < 1d 8h 0m 0s)

so you said > no need for snapshot on the datasets of Replica Target!!!! but above its blaming that it cannot compare and it has no snap to compare. so it seems it requires

syncoid root@onehost-ty-0009:pool1 pool1/backup/onehost-ty-0012 --no-stream --no-privilege-elevation --recursive --no-sync-snap --compress none

i tested also without the --no-stream and --no-sync-snap (not working) its keep blaming

NEWEST SNAPSHOT: syncoid_onereplica-ty-0001_2024-02-28:09:06:59-GMT00:00

CRITICAL ERROR: Target pool1/backup/onehost-ty-0012 exists but has no snapshots matching with pool1! Replication to target would require destroying existing target. Cowardly refusing to destroy your existing target.

      NOTE: Target pool1/backup/onehost-ty-0012 dataset is < 64MB used - did you mistakenly run
            `zfs create pool1/backup/onehost-ty-0012` on the target? ZFS initial
            replication must be to a NON EXISTENT DATASET, which will
            then be CREATED BY the initial replication process.

NEWEST SNAPSHOT: syncoid_onereplica-ty-0001_2024-02-28:09:07:00-GMT00:00

CRITICAL ERROR: Target pool1/backup/onehost-ty-0012/local_files exists but has no snapshots matching with pool1/local_files! Replication to target would require destroying existing target. Cowardly refusing to destroy your existing target.

      NOTE: Target pool1/backup/onehost-ty-0012/local_files dataset is < 64MB used - did you mistakenly run
            `zfs create pool1/backup/onehost-ty-0012` on the target? ZFS initial
            replication must be to a NON EXISTENT DATASET, which will
            then be CREATED BY the initial replication process.

NEWEST SNAPSHOT: syncoid_onereplica-ty-0001_2024-02-28:09:07:00-GMT00:00

CRITICAL ERROR: Target pool1/backup/onehost-ty-0012/local_images exists but has no snapshots matching with pool1/local_images! Replication to target would require destroying existing target. Cowardly refusing to destroy your existing target.

      NOTE: Target pool1/backup/onehost-ty-0012/local_images dataset is < 64MB used - did you mistakenly run
            `zfs create pool1/backup/onehost-ty-0012` on the target? ZFS initial
            replication must be to a NON EXISTENT DATASET, which will
            then be CREATED BY the initial replication process.

NEWEST SNAPSHOT: syncoid_onereplica-ty-0001_2024-02-28:09:07:00-GMT00:00

CRITICAL ERROR: Target pool1/backup/onehost-ty-0012/local_system exists but has no snapshots matching with pool1/local_system! Replication to target would require destroying existing target. Cowardly refusing to destroy your existing target.

      NOTE: Target pool1/backup/onehost-ty-0012/local_system dataset is < 64MB used - did you mistakenly run
            `zfs create pool1/backup/onehost-ty-0012` on the target? ZFS initial
            replication must be to a NON EXISTENT DATASET, which will
            then be CREATED BY the initial replication process.

So please how can i do

there is an issue somewhere and dealing with it since weeks....

Please to advice how to archive this.. the other closed issue was not of an clear help

madmax01 commented 8 months ago

when i remove --no-sync-snap then seems working better atm.....no error so far.........but its only working atm with autosnap and the removal of --no-sync-snap... so both activ.. why needed i don't know.. as when autosnap disabled its blaming no snap is there and semi would even not kick into... so seems first its checking if snap is there and then its kicking in an semi

is that per design? not sure why works only with 2 snaps

phreaker0 commented 8 months ago

With --no-sync-snap you need to ensure that you don't remove snapshots on boths sides which are needed for replication or it will fail like in your case.

And once again, the issue section is not a support forum, please direct those somewhere else: