DietPi-Sync | Add restore option, or, merge into DietPi-Backup?

[MichaIng]

Creating a feature request:

Is your feature request related to a problem? Please describe:

• Restoring from a dietpi-sync currently needs to be done manually.

Describe the solution you'd like:

• Would make sense to add a restore option.

Additional context:

• From forum: https://dietpi.com/phpbb/viewtopic.php?t=4562
• Thanks to @midnightwatcher for requesting.

[Fourdee]

@MichaIng

I've considered scrapping dietpi-sync, and upgrading dietpi-backup to support multiple source and target directories.

Its basically the same programs.

Only issue is with automated sync, and, supporting additional backup types + locations. Will take some time.

[MichaIng]

@Fourdee Jep, had the same idea, although thought the other way round, that a backup basically just a special sync 😆. Either way, makes sense to merge both 👍.

[MichaIng]

Further ideas about that:

• Extend DietPi-Sync to allow adding multiple slots.
• Settings are saved via arrays, similar to DietPi-WiFiDB, but (as I already plan to implement there), starting with only one or zero slots and allowing to add them via main menu option.
• The main menu then allows to select and add slots only. What is currently in main menu is then part of the slot menus.
• Add a restore option to slot menu, like we have now in DietPi-Backup.
• Add a remove option to slot menu.
• Add option to edit custom include/exclude settings, like we have now in DietPi-Backup.

  ---

  The CLI could look like this:

  dietpi-sync                  => Open main menu
  dietpi-sync <slot_ID>        => Open slot menu
  dietpi-sync <slot_ID> 1/-1   => Sync/Recover slot by ID non-interactively
  dietpi-sync all 1/-1         => Sync/Recover all slots non-interactively
  dietpi-sync daily            => Sync all slots with daily sync enabled non-interactively # Recover doesn't make sense here ;)

  ---

  What needs to be though about, when allowing flexible amount of slots is:

• When removing a slot with ID that is not the highest, it makes sense to reorder the settings array, so there will be no gaps that can grow unlimited theoretically.
• Either we could loop through existing slot IDs, but add them to config file by starting a new index from zero. Then unset all arrays and re-source the settings file.
• Or we use associative arrays, e.g. with the source path as index. Of course check and allow only one slot each source, which anyway makes sense, as it does for target.
• The include/exclude file needs to be saved per-slot as well. Also here to not mess with changing numerical array indices, we could use sub dirs like: /var/lib/dietpi/dietpi-sync/${FP_SOURCE_CURRENT//\//_}/custom_filter. This could be used of course as well for other slot settings, even for skipping the settings arrays, but only use/source within a certain slot menu. But I think for performance and I/O reasons this should be kept in one file in RAMdisk. This also allows the daily cron to easier check whether dietpi-sync needs to be started or not.

  ---

  To implement DietPi-Backup, we could hardcode a system slot, where one cannot edit the source dir or toggle delete mode.

• Additional CLI options: dietpi-sync system 1/-1
• Create compatibility script and keep alias: echo -e '#!/bin/bash\n/DietPi/dietpi/dietpi-sync system "$@"' > /DietPi/dietpi/dietpi-backup
  • This can be sadly not achieved via alias or global function, since aliases are not available in scripts and global functions not outside of interactive shells, or if actively sourced. To assure all user scripts still work, the above script needs to be placed.
  • But inform users about new CLI, so we can remove compatibility parts in the future.

    ---

    Actually I would again make userdata sync separate then, e.g. also as additional hardcoded slot. E.g. if one has Nextcloud data, music video and download folders in use and breaks the system, it is nearly no option to recover a backup, since new userdata, possible large downloads, Videos and critical Nextcloud shared data will be removed or overwritten with outdated ones. Userdata recovery most likely is used only, if not the system, but the userdata drive is broken.

    ---

    Of course we could also make all of this to dietpi-backup, since this is for sure the more used script. Only reason I thought about using dietpi-sync, is that not all syncs are meant to be backups. Perhaps users use it to keep an external drive/stick with media for holidays up-to-date before doing vacation or music collection on their mobile phone and such. There the word "backup" simply does not fit very well, while every backup is just a sync with a special purpose 😉.

[LexiconCode]

Why not make UrBackup Server a part of the install with a its Linux client. Well it's not as system resource friendly dietpi-backup but it's incredibly robust and reliable which is what you want with the backup system.