SpaceinvaderOne / Unraid_ZFS_Dataset_Snapshot_and_Replications

106 stars 23 forks source link

RSync backups not being deleted along with ZFS Snapshots #48

Closed rjwaters147 closed 1 week ago

rjwaters147 commented 1 month ago

I've been using this script for a while and I absolutely love it! However, I've encountered a small issue that I hope I can get help with.

It seems that my rsync incremental backups are not being removed in accordance with the retention policy defined for the ZFS snapshots. Specifically, I store my backups in /mnt/user/backup/rsync, and I’ve noticed that dated backups going all the way back to 03/24 are still present.

I would like to have my rsync folders follow the same retention policy as the snapshots. Essentially, when a snapshot is deleted according to the retention rules, the corresponding rsync folder should also be removed.

Is it possible to modify the script so that the rsync folders are deleted at the same time as their corresponding snapshots? Any guidance or modifications you can provide would be greatly appreciated.

Here's the user-defined variables that I use:

# Main Variables
#
####################
#
#Unraid notifications during process (sent to Unraid gui etc)
notification_type="error"  # set to "all" for both success & failure, to "error"for only failure or "none" for no notices to be sent.
notify_tune="yes"  # as well as a notifiction, if sucessful it will play the Mario "achievment tune" or failure StarWars imperial march tune on beep speaker!
                   # sometimes good to have an audiable notification!! Set to "no" for silence. (this function your server needs a beep speaker)
#
####################
# Source for snapshotting and/or replication
source_pool="cache"  #this is the zpool in which your source dataset resides (note the does NOT start with /mnt/)
source_dataset="appdata"   #this is the name of the dataset you want to snapshot and/or replicate
                                #If using auto snapshots souce pool CAN NOT contain spaces. This is because sanoid config doesnt handle them
#
####################
#
#zfs snapshot settings
autosnapshots="yes" # set to "yes" to have script auto snapshot your source dataset. Set to "no" to skip snapshotting.
#
#snapshot retention policy (default below works well, but change to suit)
snapshot_hours="0"
snapshot_days="4"
snapshot_weeks="3"
snapshot_months="2"
snapshot_years="0"
#
####################
#
# remote server variables (leave as is (set to "no") if not backing up to another server)
destination_remote="no" # set to "no" for local backup to "yes" for a remote backup (remote location should be accessable paired with ssh with shared keys)
remote_user="root"  #remote user (an Unraid server will be root)
remote_server="10.10.20.197" #remote servers name or ip
#
####################
#
### replication settings
#
replication="rsync"   #this is set to the method for how you want to have the sourcedataset replicated - "zfs" , "rsync" or "none"
#
##########
# zfs replication variables. You do NOT need these if replication set to "rsync" or "none"
destination_pool="dest_zfs_pool_name"  #this is the zpool in which your destination dataset will be created
parent_destination_dataset="dest_dataset_name" #this is the parent dataset in which a child dataset will be created containing the replicated data (zfs replication)
# For ZFS replication syncoid is used. The below variable sets some options for that.
# "strict-mirror" Strict mirroring that both mirrors the source and repairs mismatches (uses --force-delete flag).This will delete snapshots in the destination which are not in the source.
# "basic" Basic replication without any additional flags will not delete snapshots in destination if not in the source
syncoid_mode="strict-mirror"
#
##########
#
# rsync replication variables. You do not need these if replication set to zfs or no
parent_destination_folder="/mnt/user/backup/rsync" # This is the parent directory in which a child directory will be created containing the replicated data (rsync)
rsync_type="incremental" # set to "incremental" for dated incremental backups or "mirror" for mirrored backups
#
nodeOfCode commented 1 month ago

If you've changed retention parameters, the old settings can still be in effect. Solution

rjwaters147 commented 1 month ago

If you've changed retention parameters, the old settings can still be in effect. Solution

Thank you for your response and I did see that prior to posting. I didn't think I changed the parameters but I double checked just to be sure.

Here's the sanoid.conf file from /mnt/user/system/sanoid/cache_appdata/

[cache/appdata]
use_template = production
recursive = yes

[template_production]
hourly = 0
daily = 4
weekly = 3
monthly = 2
yearly = 0
autosnap = yes
autoprune = yes

I haven't made really any adjustments to the script besides updating it a few months ago after the "Merge pull request https://github.com/SpaceinvaderOne/Unraid_ZFS_Dataset_Snapshot_and_Replications/pull/29 from bobokun/patch-1" was added.

I noticed this issue awhile ago and I've been putting off troubleshooting it and I certainly have enough storage so that's not an issue. I just simply don't need that long of retention and the rsync folders just incrementing forever will inevitably become an issue even if it won't be in the near future.