psy0rz
commented
3 months ago For point 3 try zfs-autobackup v3.3 beta. Maybe this also fixes 6, but maybe thats just a truenas issue?
Open M4stakilla opened 3 months ago
psy0rz
commented
3 months ago For point 3 try zfs-autobackup v3.3 beta. Maybe this also fixes 6, but maybe thats just a truenas issue?
M4stakilla
commented
3 months ago Hi psy0rz,
Thanks a lot for your response. I do agree it is a bit tricky regarding whose issue this is (TrueNAS or zfs-autobackup). I will try to raise a TrueNAS bugreport for this as well later (but I fear they'll just dismiss it because I'm not supposed to run external scripts to do this on "an appliance").
I actually did do all my latest testing with v3.3 beta 3. zfs-autobackup_debug.log
Attached is the output of my 2 --debug runs out of which I've extracted all manual commands to perform my testing.
'3.' wasn't fixed in this version I'm afraid (see the first run - it only complained that the dataset was modified, but doesn't roll it back). '6.' can only be done if '5.' is done first, so I'm afraid also this isn't resolved.
Thanks a lot for your hard work btw. Much appreciated. I first started scripting this myself in bash, but I could have never done it as good as you've done over the years ;)
I did create a bash wrapper script already that works around issue 5. and 6. Issue 3. I cannot work around, as it happens in the middle of your script (but also only occurs the first initial replication)
Example below:
#!/usr/bin/bash
DIRECTION="master_to_backup"
SCOPE="all_snapshots"
TASK="${DIRECTION}_${SCOPE}"
SSH_CONFIG_FILE="../../../.ssh/config"
SSH_TARGET="truenas-master"
REMOTE_CMD="ssh -F ${SSH_CONFIG_FILE} ${SSH_TARGET}"
ZFS_AUTOBACKUP_FOLDER="zfs_autobackup-latest"
LOG_FILE="../../../logs/backup/zfs_autobackup-${TASK}.$(date +"%Y-%m-%d_%H-%M").log"
ZFS_AUTOBACKUP_COMMAND="autobackup-venv/bin/python -m zfs_autobackup.ZfsAutobackup"
SNAPSHOT_OPTARGS="--rollback --keep-source=0 --keep-target=0 --allow-empty --snapshot-format {}-%Y-%m-%d_%H-%M"
ZFS_OPTARGS="--zfs-compressed --decrypt --clear-refreservation"
ZFS_AUTOBACKUP_OPTARGS="--strip-path 2 --exclude-received"
cd "$(dirname -- "${BASH_SOURCE[0]}")/${ZFS_AUTOBACKUP_FOLDER}"
if [[ "${DIRECTION}" == "backup_to_master" ]]; then
SSH_OPTARGS="--ssh-config "${SSH_CONFIG_FILE}" --ssh-target ${SSH_TARGET}"
TARGET_PARENT_DATASET="master-pool/encrypted-ds"
IMPACTED_DATASETS="$(zfs list -H | awk '{print $1}' | xargs zfs get all | grep " autobackup:${TASK} " | awk '{print $1}' | xargs basename -a 2>/dev/null)"
UMOUNT_CMD="$(for IMPACTED_DATASET in ${IMPACTED_DATASETS}; do echo -n "${REMOTE_CMD} 'zfs umount ${TARGET_PARENT_DATASET}/${IMPACTED_DATASET}' ; "; done)"
MOUNT_CMD="$(for IMPACTED_DATASET in ${IMPACTED_DATASETS}; do echo -n "${REMOTE_CMD} 'zfs mount ${TARGET_PARENT_DATASET}/${IMPACTED_DATASET}' ; "; done)"
elif [[ "${DIRECTION}" == "master_to_backup" ]]; then
SSH_OPTARGS="--ssh-config "${SSH_CONFIG_FILE}" --ssh-source ${SSH_TARGET}"
TARGET_PARENT_DATASET="backup-pool/encrypted-ds"
IMPACTED_DATASETS="$(${REMOTE_CMD} "zfs list -H | awk '{print \$1}' | xargs zfs get all | grep \" autobackup:${TASK} \" | awk '{print \$1}' | xargs basename -a 2>/dev/null")"
UMOUNT_CMD="$(for IMPACTED_DATASET in ${IMPACTED_DATASETS}; do echo -n "zfs umount ${TARGET_PARENT_DATASET}/${IMPACTED_DATASET} ; "; done)"
MOUNT_CMD="$(for IMPACTED_DATASET in ${IMPACTED_DATASETS}; do echo -n "zfs mount ${TARGET_PARENT_DATASET}/${IMPACTED_DATASET} ; "; done)"
else
echo "'${DIRECTION}' is not a valid \$DIRECTION!"
exit 1
fi
if [[ "${SCOPE}" == "all_snapshots" ]]; then
ZFS_AUTOBACKUP_TASK_OPTARGS="--other-snapshots ${TASK} ${TARGET_PARENT_DATASET}"
elif [[ "${SCOPE}" == "latest_snapshot_only" ]]; then
ZFS_AUTOBACKUP_TASK_OPTARGS=" ${TASK} ${TARGET_PARENT_DATASET}"
else
echo "'${SCOPE}' is not a valid \$SCOPE!"
exit 1
fi
[[ ! -d "$(dirname "${LOG_FILE}")" ]] && mkdir "$(dirname "${LOG_FILE}")"
echo "zfs-autobackup command = ${ZFS_AUTOBACKUP_COMMAND} --verbose ${SSH_OPTARGS} ${SNAPSHOT_OPTARGS} ${ZFS_OPTARGS} ${ZFS_AUTOBACKUP_OPTARGS} ${ZFS_AUTOBACKUP_TASK_OPTARGS}" >"${LOG_FILE}"
echo "umount command = ${UMOUNT_CMD}" >>"${LOG_FILE}"
echo "mount command = ${MOUNT_CMD}" >>"${LOG_FILE}"
echo >>"${LOG_FILE}"
nohup /usr/bin/bash -c \
"{ ${ZFS_AUTOBACKUP_COMMAND} --verbose ${SSH_OPTARGS} ${SNAPSHOT_OPTARGS} ${ZFS_OPTARGS} ${ZFS_AUTOBACKUP_OPTARGS} ${ZFS_AUTOBACKUP_TASK_OPTARGS} ; ${UMOUNT_CMD} ${MOUNT_CMD} }" \
>>"${LOG_FILE}" 2>&1 \
& tail -f "${LOG_FILE}"Try setting canmount=noauto on the target datasets to see if this solves the issue. zfs problaby tries to mount the datasets, and creates a mountpoint directory in a higher dataset. (e.g. https://github.com/psy0rz/zfs_autobackup/wiki/Mounting#target-side)
Also use the --clear-mountpoint option for zfs-autobackup to automatically set canmount=noauto.
Hi,
I'm using your script on TrueNAS Scale Dragonfish and found 2 issues when replicating to a mounted dataset.
After running the script with --debug and trying all commands one by one manually, I was able to figure out the missing commands to make things work properly. This is the zfs_autobackup command I used for my replication:
root@truenas-backup:~# autobackup-venv/bin/python -m zfs_autobackup.ZfsAutobackup --verbose --debug --ssh-config ../../../.ssh/config --ssh-target truenas-master --rollback --keep-source=0 --keep-target=0 --allow-empty --snapshot-format {}-%Y-%m-%d_%H-%M --zfs-compressed --decrypt --clear-refreservation --strip-path 2 --exclude-received --other-snapshots test master-pool/encrypted-dsBelow a summary of the individual commands that are required to make it work on TrueNAS (I skipped all the snapshot, hold, release commands during my testing, as they are not relevant for these issues):
Below the 3 errors that TrueNAS throws when not remounting the dataset:
[EFAULT] Failed retreiving USER quotas for master-pool/encrypted-ds/test-ds
[EFAULT] Failed retreiving GROUP quotas for master-pool/encrypted-ds/test-ds
[ENOENT] Path /mnt/master-pool/encrypted-ds/test-ds not found