Expose snapshots to users using a wildcard entry in External Storage settings.

albert-a commented 6 months ago

How to use GitHub

Please use the 👍 reaction to show that you are interested into the same feature.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Is your feature request related to a problem?

I'm experiencing difficulties with exposing backups to users. I use ZFS filesystem snapshots in my backup solution as thay are very efficient. I want to expose this snapshots to users, but the problem is that we have folder of snapshots each of which contain all user folders. But in order to expose snapshots to particular user we need a folder of user folder versions from different snapshots. So, not to expose other users' data either we have to reorganise the structure of the snapshots, which is complicated as they are read-only, or selectively add user folders from different snapshots as an external storage entries, which is what I propose.

Describe the solution you'd like

I propose to automate adding external storage entries for user folder in each snapshots by introducing wildcard storage paths. At least for Local and SMB/CIFS storage. For example for let's talk about the Local storage. Assume that one single part in the middle of the path in Location field can be *:

Location: /mnt/home/.zfs/snapshot/*/$user

And in the Folder name can contain $match placeholder:

Folder name: /.BACKUP/$match

*How this should work Nextcloud should watch the part of Location field path up to the asterisk, (i. e. /mnt/home/.zfs/snapshot) and create dynamic entry for each subfolder in that folder, assigning the subfolder name to the $match placeholder. Thus several dynamic entries will be created:

/.BACKUP/snap_2024-05-25_040001  ->  /mnt/home/.zfs/snapshot/snap_2024-05-25_040001
/.BACKUP/snap_2024-05-27_040001  ->  /mnt/home/.zfs/snapshot/snap_2024-05-27_040001
/.BACKUP/snap_2024-05-28_040001  ->  /mnt/home/.zfs/snapshot/snap_2024-05-28_040001

Thus if /*/ is encountered in the Location then NC should create the dynamic entries.

I cannot express how much useful it would be from user perspective. With one configuration entry you gain access to all the backups for all users. For example I run Nextcloud AIO in an unprivileged LXC on Proxmox. And I can create a snapshot with a single click. How convenient it would be if you could not only create a backup of NC installation, but also expose snapshot to all users. All in one go! Furthermore this can be automated with the backup solution, synced to the other servers, accessed from NC with SMB/CIFS storage backend using the same scheme. Really powerful.

Describe alternatives you've considered

Not an alternative, but enhancement. This is optional. The UX can benefit from a recover option in file/folder details, like in Versions app. It might display a List of all backups, with those in which where the given file/folder exists at the same path highlighted. Possibly a flag: Show all backups / Show only backups with existing file/folder

If the user clicks on the backup that do not contain given file/folder at the same path, then largest common existing in the backup opened (in case parent folder renamed).

As for restore, I think we should stick to standard rule: Item.ext is restored as item.snapshot_name.ext and if it exist item.snapshot_name(1).ext and so on.

Idea on restore: To do it in an efficient way we need to perform it in 3 steps:

Hardlink current item: cp -al Item.ext item.snapshot_name.ext.tmp
Rsync snapshot to a hardlink: rsync -a /path/to/snaps/item.snapshot_name.ext/ item.snapshot_name.ext.tmp
Move recovered item into place: mv item.snapshot_name.ext.tmp item.snapshot_name.ext

This has 2 benefits:

Large folders that are slightly changed will be restored instantly.
Restored items will only take up disk space to the extent that they were modified.

While when you use an ordinary copying, even if you delete the current item, the total disk space consumed by the filesystem+snapshots will rise by the size of the item.

albert-a commented 6 months ago

Here is the similar idea on UX of this feature proposed: https://help.nextcloud.com/t/full-integration-of-snapshots/135773

vovagorodok commented 3 weeks ago

One more proposal. It can be solved like https://github.com/icewind1991/files_snapshots do. But instead Version tab create new one Snapshots?

nextcloud / server