Error message verbosity: "file changed as we read it"

[p1r473]

Hello, I am trying to get raspiBackup to backup most of my PiHole installation. I was previously excluding /etc/pihole but am tired of reinstalling PiHole after every restore.

The error I am getting is: tar: /etc/pihole: file changed as we read it raspiBackup.log Is there any way to see which file in this folder is the one that changed? The error only lists the whole folder so I dont know what to exclude. I thought its the .db file but I had this line in there: DEFAULT_EXCLUDE_LIST="--exclude=/etc/pihole/*.db" I believe the wildcard should work in the tar exclude according to my research.

Is there any way to increase the verbosity of the error message to see which file in the folder is the one that changed? Rather than listing the whole folder?

Edit: Issue #666 😈

[framps]

Try option -v. Not sure whether this will help (I don't use tar). Another way to nail down the root cause is to use inotifywait. As far as I remember there was already somebody who had an issue with pihole and I finally presented an inotifywait command to use. I found a hit into my website but the text is not there any more for some reasons :angry:

[p1r473]

Thanks, command is inotifywait -m -e delete -e create -e move -e modify -e attrib <DIRECTORY>

[framps]

@p1r473 Would you please share which file or directory of pihole created your issue?

[p1r473]

/etc/pihole/pihole-FTL.db /etc/pihole/pihole-FTL.db-journal

[p1r473]

Well I tried to exclude these files but still same issue Once I ran inotifywait -m -e delete -e create -e move -e modify -e attrib /etc/pihole and removed duplicate lines, only these existed

/etc/pihole/ CREATE pihole-FTL.db-journal
/etc/pihole/ ATTRIB pihole-FTL.db-journal
/etc/pihole/ MODIFY pihole-FTL.db-journal
/etc/pihole/ MODIFY pihole-FTL.db
/etc/pihole/ DELETE pihole-FTL.db-journal

so only these 2 files:

/etc/pihole/pihole-FTL.db
/etc/pihole/pihole-FTL.db-journal

However, it seems still file changed as we read

Im excluding these 2 files:

DEFAULT_EXCLUDE_LIST="--exclude=/mnt/* --exclude=/var/lib/samba/private/msg.sock/* --exclude=/etc/pihole/*.db --exclude=/etc/pihole/*.db-journal  --exclude=/var/log/*.* --exclude=/var/log/*/*.* --exclude=/var/lib/php/sessions/* --exclude=/var/lib/unbound/root.key.* --exclude=/var/cache/* --exclude=/var/spool/postfix/* --exclude=/home/pi/.local/share/Trash/*--exclude=/root/.local/share/Trash* --exclude=/.Trash/* --exclude=/var/www/apache/public/BrownNoise.mp3"

raspiBackup.log

Sidenote: Not sure why Im getting !!! RBK0281W: Unsupported version of raspiBackup.sh.. Perhaps related to #681 ? @framps

[framps]

Perhaps related to https://github.com/framps/raspiBackup/issues/681 ?

Yes.

In the log I find

20230920-051923 DBG 2515:                      --- LC_ALL=C tar         -cpi --one-file-system              -z              -f "/mnt/@HOSTNAME@/@HOSTNAME@/@HOSTNAME@-tgz-backup-20230920-051911/@HOSTNAME@-tgz-backup-20230920-051911.tgz"         --warning=no-xdev       --numeric-owner         --exclude="/mnt/@HOSTNAME@/*"       --exclude="/tmp/raspiBackup.log"        --exclude="/tmp/raspiBackup.msg"        --exclude='.gvfs'       --exclude=/proc/*       --exclude=/lost+found/*         --exclude=/sys/*        --exclude=/dev/*        --exclude=/tmp/*        --exclude=/boot/*       --exclude=/run/*        --exclude=/media/*      --exclude=/mnt/* --exclude=/var/lib/samba/private/msg.sock/* --exclude=/etc/pihole/*.db --exclude=/etc/pihole/*.db-journal --exclude=/etc/pihole/versions --exclude=/var/log/*.* --exclude=/var/log/*/*.* --exclude=/var/lib/php/sessions/* --exclude=/var/lib/unbound/root.key.* --exclude=/var/cache/* --exclude=/var/spool/postfix/* --exclude=/home/@USER@/.local/share/Trash/*--exclude=/root/.local/share/Trash* --exclude=/.Trash/* --exclude=/var/www/apache/public/BrownNoise.mp3      /
tar: /etc/pihole: file changed as we read it
tar: /etc/pihole: file changed as we read it

Your exclude statements are correct.

I just created a tar backup of my system running pihole (I use rsync all the time). I don't stop any pihole nor do I exclude any files but the tar backup finished successfully.

Is there any reason why you don't stop pihole in raspiBackup? If yes I suggest to exclude the whole pihole directory and check.

[p1r473]

I try to not stop pihole as it is running my DNS server and do not want to cause any DNS related network outage It does work fine if I exclude the whole /etc/pihole dir

[framps]

I try to not stop pihole as it is running my DNS server and do not want to cause any DNS related network outage

I see your point :smile:

I create my backups at 5am where there are usually no clients used by humans active. All my servers which need network connectivity all the time are configured not to use pihole but my internet router DNS. So I would be able to stop pihole without interrupting my servers.

But again: I can create a backup without any issues with pihole if I don't stop pihole. Looks like you have some very special environment setup :wink:

[p1r473]

The correct command to see all file activity is as follows. I added more toggles than just the one above inotifywait -m -e delete -e create -e move -e modify -e attrib -e access -e close_write -e close_nowrite -e close -e open -e moved_to -e moved_from -e move_self -e delete_self -e unmount /etc/pihole/

With this I was able to see some non standard custom files and folders of mine being accessed

[framps]

Thank you very much for sharing :+1:

Looks like it's a good idea to include all events instead of a subset to find the responsible object which changed during backup :smile: .

[framps]

Notification eMail from github from p1r473: I have no clue why I can't find the comment here so I add it on behalf of p1ra73:

I've still been fighting with this on my pihole.

Ive been trying everything to see whats going on in /etc/pihole. It seems the only things moving are

/etc/pihole/pihole-FTL.db
/etc/pihole/gravity.db
/etc/pihole/pihole-FTL.db-journal

I found a wordaround you might want to know.
DEFAULT_EXCLUDE_LIST="--ignore-failed-read --warning=no-file-changed .....
This just outright ignores the errors. Has worked for my last 2 backups.

@p1r473 Thank you very much for sharing.

For your info: Actually option DEFAULT_RSYNC_BACKUP_ADDITIONAL_OPTIONS is supposed to be used to add any additional rsync options :wink: But DEFAULT_EXCLUDE_LIST adds also the additional options to the backup call. So both options have the same effect.