DietPi-Backup | Allow auto-restore from backup on NFS share

[alexiri]

Creating a feature request

Is your feature request related to a problem? Please describe:

• Headless restoring of a system from a backup could be a lot simpler than it currently is.

Describe the solution you'd like:

• A DietPi backup should include an up-to-date and fully-configured dietpi.txt and dietpi-wifi.txt. This would include, for example:
  • Current network configuration of the machine
  • Samba/NFS configuration (if needed to reach the backup archive)
  • Headless install enabled via AUTO_SETUP_AUTOMATED=1
  • AUTO_SETUP_BACKUP_RESTORE configured for the current machine's backup location
• Those two generated files could be copied to the /boot partition of a fresh image, and when the RPI boots it could restore itself from the backup

Describe alternatives you've considered:

• Currently, you can get the original dietpi.txt and dietpi-wifi.txt from a backup, but a system's configuration will have diverged since it was originally installed. These files still need to be edited to restore from a backup.

[Joulinar]

Not sure if I understood your request fully. A restore will recover your system to the exact same state as it was the moment you have taken the backup. This includes the entire system and configuration files. It will not reset the system into an initial status to perform an initial setup.

dietpi-wifi.txt will be used on a first initial setup only. It is not present/used on a later stage if initial setup has been completed.

[alexiri]

Yes, but in order to perform the restore from a new system image, I first will have to configure dietpi.txt and dietpi-wifi.txt correctly so the new RPI can access the network and reach the restore archive. What I would like is for the backup process itself to generate these two config files automatically so that all I have to do is copy them to the system image and the new RPI will have everything it needs to restore itself.

[Joulinar]

If you have a backup attached via USB, the restore could be triggered right at the beginning without any configuration needed. You just need to set the correct value inside dietpi.txt on a fresh image before first boot.

[alexiri]

Yes, I know, I'm not saying it's not possible to restore from a backup, I'm just saying it could be even easier.

The necessary changes to dietpi.txt could be generated automatically during the backup process, when all the needed information is already known. This would be especially useful when backups are stored on a remote NFS share, as is my case. Why should I have to edit dietpi.txt manually to set a hostname, network configuration, NFS mount, restore location, etc? The backup process already knows all that information, it could generate a valid dietpi.txt for me. Then all I would need to do is flash the dietpi image, copy the generated dietpi.txt and boot.

[Joulinar]

Well there is no NFS client software present at the beginning. Means you would need to complete the initial setup before being able to restore a backup from a NFS share. There is no way to restore configuration files beforehand. Therefore network setup would need to be done manually before initial boot happen.

[alexiri]

The NFS/SMB install could be done as part of a AUTO_SETUP_CUSTOM_SCRIPT_EXEC, that again the backup process could generate automatically.

I'm not sure I understand your hesitation on this: we can already install and configure software through the headless install feature during first boot, why not restore from a remote share as well? Why do you think this wouldn't be useful?

[Joulinar]

Because configuration files would be needed as first step. They are needed to startup the system before you are reaching the state where software is installed and configured. There order in which thinks happen matters. One of the early steps is to configuration network. This happen way before any NFS client software would be installed.

[alexiri]

Yes, that's clear. What I'm saying is that the backup process has all the necessary information to automatically generate the configuration files (and custom scripts) that would be needed for an unattended restore from an NFS share (or any other location). Is this assertion incorrect? Am I missing something?

[Joulinar]

Where should the backup script get the information from? These information are located on a share that is not accessible at the beginning without any network configuration done beforehand. You still need to configure network as one of the first step before being able to get any information from a remote share.

[MichaIng]

Of course it would be useful, see the list of open issues, full with awesome ideas 😃. We are seeing this from the developers point of view, at which point we'd be able to implement this without days of code rework and in a way that works reliably and intuitive.

The automatic backup restore currently runs before any software is installed, so as stated, no NFS/SMB client is available. AUTO_SETUP_CUSTOM_SCRIPT_EXEC runs even later, as very last step of the first run setup, and it is not meant to be used for software installs, AUTO_SETUP_INSTALL_SOFTWARE_ID is. So we'd need to change the order, which means more overhead for anyone who is not restoring a backup from network drive, and, not sure whether we should somehow set the AUTO_SETUP_INSTALL_SOFTWARE_ID based on the network share detection during backup target change?

But the bigger problem is that we'd need to know the NFS/SMB server hostname/IP, paths/shares and credentials, which means a lot of additional dietpi.txt (?) entries that currently do not exist and that would need to be generated/maintained on every backup target change for just the case that someone plans do to an automated restore from that same network drive. It also causes a security downside, as we store SMB credentials in a dedicated file, readable only by root, and I'm not keen to store those in plain text into the world-readable dietpi.txt 🤔.

And indeed network setup still needs to be done, if it is not simply Ethernet + DHCP, a similar issue with this is that I'm not keen to leave the WiFi key (or even EAP info) in dietpi-wifi.txt world-readable just to cover such case.

---

So generally the idea/feature would be nice, but the practical implementation means a lot of rework, detailed thoughts and either way comes with negative security impacts. So the question is how much users would really use it, and whether e.g. we'd then need to have it behind a setting which clearly states that in case WiFi and SMB credentials are stored world-readable in /boot.

[MichaIng]

Ah wait, backups cannot be done to SMB anyway, and NFS has no credentials. So as far as I can think of it now:

• Have an option in dietpi-backup for automatic NFS share restore on first boot.
• If set, e.g. save the related line from /etc/fstab in dietpi.txt.
• If WiFi is used, additionally save the WiFi SSID/key database as /boot/dietpi-wifi.txt. At least on non-RPi systems with no FAT /boot filesystem it can be made root-only readable, on FAT however this does not work.
• Run auto-restore after first run installs have been done, set AUTO_SETUP_INSTALL_SOFTWARE_ID=110 accordingly, remove all other AUTO_SETUP_INSTALL_SOFTWARE_ID entries to avoid overhead when the very first DietPi install was automated with software installs already?

Aside of this, there are indeed a few dietpi.txt entries we could keep in sync in general, like the hostname and network details (DHCP, static IP info).

---

EDIT

Ah, or are you saying these files/info should be stored at the backup location? This would solve the security issues, respectively it's then up to the user to take care access permissions on the server side.

[alexiri]

Yes, I'd store these files as part of the backup itself, that's what I tried to express in the first message. Sorry if I didn't express myself well, but I'm glad to see we're now converging. :)

As you've pointed out, there wouldn't be any (additional) security issues, as the backup itself already includes all the secrets.

I think the approach you outlined in the last message makes sense and wouldn't require any changes to the order of operations of the current installer (I think). You could configure the network in dietpi.txt, AUTO_SETUP_INSTALL_SOFTWARE_ID=110 and then have a short AUTO_SETUP_CUSTOM_SCRIPT_EXEC script that mounts the share and runs dietpi-backup to start the restore.

[MichaIng]

as the backup itself already includes all the secrets.

But not world-readable. However, yes, since the backup location must support UNIX permissions, we can keep it like that.

and then have a short AUTO_SETUP_CUSTOM_SCRIPT_EXEC script that mounts the share and runs dietpi-backup to start the restore.

We must not mix native features with the custom script. The auto-backup restore is done as hard code in dietpi-software and it must stay like that. So this still implies a reorder of the first run setup steps pre/post software installs.

[alexiri]

Ok, I understand. That is the cleaner approach, I suppose. I was just thinking of ways to implement it with what is already there, dietpi-software is already quite a beast.