jimsalterjrs / sanoid

These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
http://www.openoid.net/products/
GNU General Public License v3.0
3.1k stars 303 forks source link

Ignore unmodified datasets #824

Closed Thalhammer closed 1 year ago

Thalhammer commented 1 year ago

Hi, I have a lot of datasets, which often remain unmodified for extended periods of time, but should be fairly frequently snapshoted when used (currently in 15 minute intervals). This causes tons of empty (0B used) snapshots which make viewing the list harder and might even slow zfs down a bit.

Would it be possible to ignore doing a snapshot if theres not been any write activity to the filesystem since the last snapshot ?

MynaITLabs commented 1 year ago

Or at least an option in 'findoid' and 'zfs list -t snapshot' to ignore/not-display snapshots which have 0 byte "USED" differences. something like "--ignore-zero-used" ?

Note: Before feature is implemented. Could also write an 'awk' script to not display lines with 2nd column "USED" being exactly " 0K ".

jimsalterjrs commented 1 year ago

Would it be possible to ignore doing a snapshot if theres not been any write activity to the filesystem since the last snapshot ?

No, because it would make the accounting on keeping the correct number of dailies, monthlies, etc obnoxious... And even more importantly, would mean that if you wanted to roll back to last Friday's daily, there might not BE a daily from last Friday, if nothing changed between the most recent hourly (or frequently!) and 23:59 on last Friday when the daily should have been taken.

This is a WONTFIX for the project, but you can easily create your own "cleanup" script that destroys any snapshot with 0 bytes in the USED column of zfs list -t snap.

glibg10b commented 3 months ago

This script returns 1 if the dataset hasn't changed since the last snapshot. It needs the libzfs module which is maintained by TrueNAS and can be installed with sudo apt install python3-libzfs on Ubuntu.

#!/bin/python3

import libzfs
import os

def exit_success_if_changed(dataset_name):
    dataset = libzfs.ZFS().get_dataset(dataset_name)
    written = dataset.properties['written'].value

    if written != '0':
        exit(0)

for dataset in os.environ['SANOID_TARGETS'].split(','):
    exit_success_if_changed(dataset)

exit(1)

To use it, add the following to /etc/sanoid/sanoid.conf:

pre_snapshot_script = /path/to/script.py
no_inconsistent_snapshot = true