Backup of freshly installed system fail - No Permission to create backup folders

[kkoletz]

Hi Dear Community,

I've freshly installed a few programs from the IOTStack(pihole, grapafan, influxdb, tasmota, portainer_ce) and wanted to test the backup function. Unfortunately when I run from the menu: Backup and Restore > Run backup the following errors appear: Execute Backup: mkdir: cannot create directory ‘./backups/logs’: Permission denied mkdir: cannot create directory ‘./backups/backup’: Permission denied mkdir: cannot create directory ‘./backups/rolling’: Permission denied touch: cannot touch './backups/logs/backup_2021-01-22_0058.log': No such file or directory ./scripts/backup.sh: line 70: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 71: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 72: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 73: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 74: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 77: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 81: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 86: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 87: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 88: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 100: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 104: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory cp: cannot stat './.tmp/backup/backup_2021-01-22_0058.tar.gz': No such file or directory cp: cannot stat './.tmp/backup/backup_2021-01-22_0058.tar.gz': No such file or directory ./scripts/backup.sh: line 117: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory stat: cannot stat './.tmp/backup/backup_2021-01-22_0058.tar.gz': No such file or directory ./scripts/backup.sh: line 121: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 122: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 124: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 125: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 126: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 128: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 129: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory Something went wrong backing up. The temporary backup file doesn't exist. No temporary files were removed Files: ./.tmp/backup-list_2021-01-22_0058.txt ./scripts/backup.sh: line 145: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 146: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory ./scripts/backup.sh: line 147: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory cat: ./backups/logs/backup_2021-01-22_0058.log: No such file or directory

Backup completed. Press [Up] or [Down] arrow key to show the menu if it has scrolled too far.

When I look into the IOTstack/backups/ folder I see a folder for the influxdb and that's all. Any idea what's wrong and how to solve this?

Thanks a lot!

[Paraphraser]

Can I request the output from running the following commands, please:

$ ls -ld ~/IOTstack
$ ls -l ~/IOTstack/backups ~/IOTstack/backups/influxdb

[Paraphraser]

By the way, I came up with an alternative backup/restore strategy. If you want to consider it, please take a look at Paraphraser/IOTstackBackup.

[kkoletz]

Sure, here it is the IOTstack folder rights: pi@raspberrypi:~ $ ls -ld ~/IOTstack drwxr-xr-x 13 pi pi 4096 Jan 22 00:53 /home/pi/IOTstack and here the influxdb one: pi@raspberrypi:~ $ ls -l ~/IOTstack/backups ~/IOTstack/backups/influxdb /home/pi/IOTstack/backups: total 4 drwxr-xr-x 3 root root 4096 Jan 22 00:53 influxdb

/home/pi/IOTstack/backups/influxdb: total 4 drwxr-xr-x 2 root root 4096 Jan 22 00:53 db

Thanks for your help!

[Paraphraser]

Blast! Finger trouble at my end. Sorry. I meant to put the "a" flag on the second one so I'd also get to see "." for the backups folder. How about ls -ld ~/IOTstack/backups too?

IOTstack needs to be owned by pi ✅ backups should also be owned by pi (so we still need to check that) ❓ influxdb should be owned by root ✅ db should be owned by root ✅

The error messages in your original post were talking about creating sub folders in backups and those were what were failing so we have probably narrowed it down to ~/IOTstack/backups being owned by root while the script is expecting pi.

If it turns out to be the case that backups is owned by root, you'll obviously need to fix that (chown pi:pi ~/IOTstack/backups) but, unless you can think of a reason why it might've come to be owned by root, you'd be well advised to check your whole IOTstack structure. Aside from the folders already mentioned inside backups, the only other thing that should be owned by root is volumes and most, but not all, of its contents. Mosquitto is an example of an exception (owned by user ID 1883) but there's no hard and fast rule that can be applied here.

[kkoletz]

@Paraphraser Thank you a lot! Indeed somehow the backups folder was owned by root. Doing the chown pi:pi ~/IOTstack/backups solved the problem. Now I can do backup and it worked. Also in order to test if the restore is also there before I can confidently use it, I also tried to do the restore. I copied the backup tar.gz file in the backups directory and then renamed it to backup.tar.gz. After restore was successfully done, I tried to access pi-hole and it showed some errors that the the pihole database is not found and so on. SOLUTION: Seems restarting the Raspi helps, so that the new config and database files for pi-hole are regenerated.

Thanks for your help!

[Paraphraser]

The link I gave you earlier to my own backup/restore scripts is how I do all of my systems. Although I started from the official gcgarner/IOTstack backup script (there was no restore script at the time), it's a long time since I looked at the current "official" scripts. I mention this so you understand where I'm coming from in what I say next.

In my view it is most unwise to attempt a restore while any part of IOTstack is running. What is happening is that a large number of files that are open (ie in use by processes running inside containers) are being replaced on-the-fly. Each replaced file's inode will change so even though the file is same name at same path and may even have the same content, it is a different file. Under the hood inside containers this may well cause a fair bit of crashing and burning. It's hardly surprising that PiHole objected. What's more surprising is that nothing else objected forcefully (ie so as you noticed it).

If you look at my restore script, you'll see that the first thing it does is check whether any part of the stack is running and then sends a "down". Then it loads the contents of what I call the "general" backup (everything that is copy-safe). That completely replaces services and volumes.

Then it loads the influx portable backup into backups/influxdb/db. At that point, the side-effect of the general restore replacing volumes is that influxdb doesn't even have a folder inside volumes. The script tells the influxdb container to "up". docker-compose sees that influxdb's folders inside volumes don't exist and creates empty folders. Then influxd starts and sees empty folders so it initialises new empty databases. The script sends the command to tell influx to load the portable backup and that repopulates the databases. Then the container is taken down again. At the very end, the main restore script remembers if it had to take the stack down at the beginning and brings it up again.

When I "grep" the current offical restore scripts for evidence of taking the stack down, I don't see anything. I haven't done an exhaustive check so I might be wrong. But if I'm right then the onus is on the poor user (ie you) to remember to take the stack down before a restore. You don't have to worry (as much) about running backups while the stack is up. The sole proviso is that anything not copy-safe is handled properly. That mostly applies to database packages like influxdb, Postgres, MariaDB, etc. I only run influxdb which is why that's the only non-copy-safe thing I backup.

SQLIte is usually copy-safe, although you can get caught out if an app opens multiple SQLite DBs and issues transactions that write across databases.

PiHole uses SQLIte databases but it seems to treat them as independent entities so it meets the definition of copy-safe.

I also run PiHole. I treat it as copy-safe in my backup routine and it restores without so much as a blip. But, then, my stack is always down while the restore is happening...

Subtle as a sledgehammer, right? 😎

[kkoletz]

Hi @Paraphraser Thanks a lot for your clarification. I got your point. Next time I do a restore of the system (in case it's needed), then I will definitely stop all the "to be restored" instances and then start them again after the restore is done. Just a simple question, would it be possible to announce the author of the IOTstack script to enhance it with some features from your restore/backup script version?

Thank you!

[Paraphraser]

If you think some features should be in the official script then the best way to go about it is to prepare a Pull Request. You're welcome to use anything from my scripts if you want to do that.

The main reason I went my own way was that I really did not want to get into the game of being "Mr Backup" for all containers. Making sure Influx was working was hard enough but I needed that to work so I had the motivation, the wherewithal to test it, and the need to keep it going as and when IOTstack and/or InfluxDB change. I could not offer the same level of support to Postgres, MariaDB or any other container that turned out to need special handling.

Backup/restore is far too important to be giving people false comfort that you really know what you're doing. The only thing I can really say with confidence is that, for the containers I run (mosquitto, influxdb, nodered, grafana, pihole, gitea, portainer-ce), I routinely both restore in-situ (ie load onto my "test" RPi any backup taken on my "live" RPi) and do bare-metal restores on a freshly-imaged RPi.

I am reasonably certain that any container where the data is copy-safe will work properly because, from time to time, I do run-up other containers and I do run them through a backup/restore cycle.

If I found a reason to run-up a container that was not copy-safe then I'd have the motivation to figure out the backup/restore side of things.

Beyond that, all bets are off, which is why the doco on my repo is at pains to lay that out in black and white.

[david-robson]

Firstly, thanks for the great work here! This is still an issue with the script now. I resolved it by changing the owner of the backups folder to pi: sudo chown -R pi:pi backups