nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.43k stars 4.07k forks source link

Equivalent of Windows folder sync on Linux. #49190

Open Bockeman opened 5 days ago

Bockeman commented 5 days ago

How to use GitHub

Is your feature request related to a problem? Please describe. I'd like the folder synchronization to work on Linux like it does on Windows.

The file system on Windows is different from that on Linux, but Nextcloud does a very good job of storing Windows data on a Linux server. Ok, so all the files on the linux server are owned by the http owner such as apache, the permissions are probably coalesced and Windows shortcuts are ignored.

Importantly, when a Windows client machine restores/fetches/updates files from the Linux server, they appear to all work seemlessly without any complaint from Windows. Under the hood, there may be differences between files created on one Windows machine, sync'd to the Linux server, then sync'd to a another Windows machine. But to the user, it just works. A world class excellent product.

Now lets suppose the client machine (desktop or laptop) is running Linux (and that SELinux is not running or is at least permissive). The usage example, equivalent to the Windows usage example below, simply does not work. It cannot, because there are so many features that Linux relies upon (ownership, permissions, symbolic links), and these are not supported over the current WebDav (or whatever) interface the Nextcloud uses.

Describe the solution you'd like I'd like the richer Linux metadata to be encoded somehow and passed over the WebDav interface. I'm not asking a Windows machine to handle this extra metadata, it should ignore it on download from the server, and not provide it for upload (at least in the first instance, though some features, like a Windows shortcut could be mapped to a Linux symbolic link and vice versa, but even that creates issues which I want to keep out of bounds for this request).

Describe alternatives you've considered The Linux "rsync" does an excellent job of keeping the laptops aligned, provided this is a one way sync, that is rsync is from powered to laptop. But even then, "rsync" is inadequate because it gets tripped up with temporary files (e.g. lock files) and temporary permission changes (only writable when lock is held). I've experimented with trying to make rsync two way (e.g. by taking turns in each direction), but this proved to be a disaster. It got even more tangled by the temporary files, and got into a complete mess if files or folders were moved/renamed. It actually turned out to be dangerous, removing folders from both ends. Ok, so my implementation may have had weaknesses, but rsync was not designed for this job. rsync does not have any conflict resolution mechanism. It is my understanding the Nextcloud handles a number of scenarios, like file in use, and does support conflict resolution.

Additional context Here is a Windows usage example. In the office, a user has a Windows desktop with multiple large screens, various devices attached or nearby, and possibly network access to data storage servers. Nextcloud client ensures that all files in selected folders is continuously sync'd with the Linux server. In addition to the typical Document, Picture, Video folders this might include more exotic folders like AppData/Roaming/Thunderbird, or even a dynamic folder such as GoogleDrive [Yep, I've been syncing shared GoogleDrive storage with Nextcloud successfully for ages]. This user has a Windows laptop, that normally stays at home or is taken to other locations. When the laptop is switched on, Nextcloud sync's all the folders, and (onced sync'd) the user can carry on where they left off in the office (barring the office based hardware). Assuming, the user checks that all files have been sync'd before powering off the laptop, upon return to the office, the user can resume on the desktop "where they left off" on the laptop. There's an underlying assumption that the user only works on one machine, the laptop or the desktop. But even in the case where the user accidently leaves an application running, such as an email client, Nextcloud detects the potential conflict and provides a very reasonable way of resolving those conflicts.

The obvious Linux usage example, is equivalent to the Windows usage example above, but all the client machines (desktop, laptop) are running an equivalent OS, Linux. I am not expecting any useful cross platform synchronisation (apart from existing trivial usage of files and folders not relying on the richer Linux file system metadata).

Here's a more challenging Linux usage example. The site has a farm of Linux machines and infrastructure. There are also a number of laptops which are normally connected. It is not cost effective to provide UPS for the whole infrastructure mainly because a power cut is just one of many scenarios that could result in loss of service. When there is a power outage, all the powered Linux machines go down, but the laptops keep running for several hours (long enough for considered manual intervention contingencies). When power is restored, all machines essentially power up to an idle state waiting for manual confirmation (this is so much safer and more convenient than the calamity that is introduced by an uncoordinated power up). Meanwhile, the laptops remain running. It these laptops that are used to coordinate bringing up the powered systems. If the laptops were running Nextcloud, then they would have an up-to-date record of essential information prior to the power cut, and would be immediately ready to coordinate bringing up the powered systems. Without Nextcloud, the bring up process is convoluted and takes hours.

I know an awful lot of work is put in to keep Nextcloud functioning correctly and providing ever more powerful features. Thank you all. Sometimes, revisiting the foundations (of the fundamental two-way sync) might not be appealling. But think of the rewards and the potential, such as sync'ing other platforms like MacOS, Android, iOS, and then eventually the cross platform syncing! (There's a messy cross platform solution where the additional metadata is stored on hidden files on any client machine that does not support particualr metadata features -- but I'm sure there's a better solution waiting for someone to invent).

MrRinkana commented 21 hours ago
  1. File-metadata such as creation date, exif and such is already synchronised, to all clients no matter os (win, linux, mac, android, ios).
  2. filesystem specific metadata such as permissions (ownership), where next block is, selinux context, eventual checksums (resilient fs:s) and such should not be synchronised, and is not to any os. Every client just creates the files with its own user:group (of the local user/machine). Its like that on both windows and unix based. It does not make sense, nor will it work to have the same user:group (or other such filesystem metadata) owning the files on all systems. Wanting that is doing something wrong.
  3. Your text is long and I at-least cant follow what you want. What exactly are you trying to do that's not working? What exactly do you want added?

To my knowledge the only practical difference between linux client and windows is the support for virtual filesystems (file only downloaded when accessed), which is a important feature, but not something you seem to be describing/asking for at all, and that is tracked elsewhere.