jimsalterjrs / sanoid

These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
http://www.openoid.net/products/
GNU General Public License v3.0
3.01k stars 297 forks source link

Snapshots have wrong time. #560

Closed abclution closed 4 years ago

abclution commented 4 years ago

Not really sure where to adjust or check this, but snapshots run with the systemd timer have the wrong time.

drwxrwxrwx 1 root root 0 Jun 15 15:31 autosnap_2020-06-15_12:28:00_frequently/ drwxrwxrwx 1 root root 0 Jun 15 15:31 autosnap_2020-06-15_12:30:00_frequently/

root@sporos /m/H/a/.z/snapshot# date Mon 15 Jun 2020 03:31:36 PM EEST

drwxrwxrwx 1 root root 0 Jun 15 15:31 autosnap_2020-06-15_12:28:00_frequently/ drwxrwxrwx 1 root root 0 Jun 15 15:31 autosnap_2020-06-15_12:30:00_frequently/ drwxrwxrwx 1 root root 0 Jun 15 15:34 autosnap_2020-06-15_12:32:00_frequently/ drwxrwxrwx 1 root root 0 Jun 15 15:34 autosnap_2020-06-15_12:34:00_frequently/ root@sporos /m/H/a/.z/snapshot# hwclock 2020-06-15 15:34:56.372733+03:00

Everything is off by 2 hours with the snapshot names, but my system time is set correctly?

Edit: Hmm, I'm seeing the opposite behavior of the issue here: https://github.com/jimsalterjrs/sanoid/issues/188

Ok, I guess the default now is set to UTC because of daylight savings nonsense.

phreaker0 commented 4 years ago

@abclution the default systemd timer unit set's the env variable TZ=UTC for running sanoid

abclution commented 4 years ago

I was trying to find out that, where and how basically where that is set, and where I could change it. But in the end after reading about the daylight savings issues I decided to go with the UTC sane defaults.

What was really throwing me for a loop is that running sanoid -cron manually was creating a localtime snapshot, but the automatic ones run by systemd were of course UTC.

I was extremely confused for awhile wtf was going on...

@phreaker0 Thanks!

spacerunner5 commented 5 months ago

Ran into the same problem here after replicating from an old gentoo system (which uses OpenRC) to a Debian host (systemd). Fixed by adjusting Environment=TZ in sanoid.service:

# file: /usr/lib/systemd/system/sanoid.service

[Unit]
Description=Snapshot ZFS Pool
Requires=zfs.target
After=zfs.target
Wants=sanoid-prune.service
Before=sanoid-prune.service
ConditionFileNotEmpty=/etc/sanoid/sanoid.conf

[Service]
Environment=TZ='Europe/Berlin'
Type=oneshot
ExecStart=/usr/sbin/sanoid --take-snapshots --verbose

Environment=TZ=\<mytimezone> was taken from https://bookstack.swigg.net/books/blackbox/page/storage-backups#bkmrk-configure-sanoid

Two questions:

  1. The question really is whether sanoid's defaults should differ from the rest of the system. Is it really necessary for the timer unit to write TZ=UTC env var? Optionally would my be better.
  2. Could Environment=TZ=XX be set elsewhere, maybe on /etc/sanoid/sanoid.conf? (in order to avoid modifying the service file)

Again: Thanks for Sanoid :smiley:

spacerunner5 commented 5 months ago

Additional Note:

Adjusting the TZ settings (see above) also solved another problem with replication.

Szenario:

Result: On the target server B, sanoid creates new yearly / monthly / weekly snapshots - although they already exist.

Without reviewing the code I assume that as seen by sanoid on Server B, the lastest snapshot is newer than system time. I just post this finding here for informational purposes.

phreaker0 commented 5 months ago

@spacerunner5 the default of UTC is the sane one. because in local time with DST you will miss an hourly snapshot per year.

You also shouldn't edit the provided sanoid service file ( /usr/lib/systemd/system/sanoid.service) directly, you should only override settings which you need changed:

systemctl edit sanoid.service

Regarding your replication comment, this seems to only be an issue for you because you are apparently snapshotting replication targets but this doesn't make sense, because the next replication those will be rolled back because the can't match with the source dataset.

spacerunner5 commented 5 months ago

@phreaker0 Thank you for the comment :)

Systemd: To be honest: As an ages gentoo user - which is still my favorite - it takes me a while to get used to systemd. However, I figured out the systemd conf overrides in the mean time while quick-and-dirty sorting out some some other issue with zfs-zed (not sanoid related).

Time Zones: I rather lose that one hourly snapshot per year rather than having my snapshots differ from my actual time.