search

nextcloud / desktop

💻 Desktop sync client for Nextcloud
https://nextcloud.com/install/#install-clients
GNU General Public License v2.0
2.91k stars 773 forks source link

LAN sync with other clients on local network #2010

Open biva opened 4 years ago

biva commented 4 years ago

Hello,

I'm in Africa and we have both a slow and expensive bandwidth (illimited plans are very rare). In addition, in a climate change perspective, it would be useful to limit bandwidth usage over large distances to save energy. In order to optimize this, we'd like to have the ability to sync over a LAN both with the server if it's on the LAN, and especially with other clients on the same LAN, who have already synced.

It is a cool and efficient existing Dropbox feature (https://help.dropbox.com/installs-integrations/sync-uploads/lan-sync-overview). And a lot of discussions have already started, both on owncloud and Nextcloud, but it looks like there's no activity anymore for years (https://github.com/nextcloud/server/issues/7092, https://github.com/nextcloud/server/issues/8331, https://github.com/owncloud/client/issues/230).

Would it be possible to consider this feature? Thank you!

For reference, other links where this feature is requested: Nextcloud:

owncloud:

Client configuration

Client version: 2.6.4stable-Win64 (build 20200303)

RogoTe commented 4 years ago

Just for the reference... Have you ever looked at syncthing.net and pydio?

michaelstingl commented 4 years ago

Just for the reference... Have you ever looked at syncthing.net and pydio?

@RogoTe maybe you could provide some more context/pointers. Syncthing doesn't have a central server, and Pydio doesn't have LAN sync? (afaik)

biva commented 4 years ago

@RogoTe I'm not a dev so I can't know if they are good strategies. I think they were already mentioned in previous posts (notably https://github.com/owncloud/client/issues/230#issuecomment-349968421). [Edit] Synchthing was also discussed in https://github.com/nextcloud/server/issues/8384

How would the Nextcloud team consider this feature request?

wizdude commented 4 years ago

Syncthing doesn't have a central server, and Pydio doesn't have LAN sync? (afaik)

Syncthing is designed to be peer to peer and your main server can most certainly be the central server. Replication occurs between the peers and the preference will be to use the local LAN where possible. Ideally I'd like to see this technique implemented into the Nextcloud desktop agent rather than supporting two different pieces of software.

zephyrus00jp commented 4 years ago

Hi, I wrote in 2015 at owncloud/client#230

I am not entirely sure what Dropbox does (especially the communication with its server: it is encrypted using SSL, I think.), but as someone mentioned that there are peer-discovery and communication. I found out about it when I checked the LAN activity for UPnP and other low-level lan activity. Wireshark is your friend.

--- That is, Dropbox uses the same underlying protocol used by UPnP to broadcast a request or announce the availability of files and then let the peers to exchange files after a certain negotiations.

Its payload seems to contain the file list and other information that tells, based on my educated guess, with server's PKI signature (to avoid forgery) [that is the client asks for the server to create such a list with the server signature], that the client has the files (timestamp, checksum) and the authority to hand out the files.

--- Maybe for the files the particular client has created/updated recently (i.e., has just uploaded to the server. Obviously, this client has the most up-to-date copy of the files.)

When the other client checks with the server and finds that it needs to download a set of files, it can ask the local LAN first to see if other PCs have such files, and when one PC answers yes with the above list (and it is signed with server's key), it can begin copying the file using SSL using a key based on RSA/DH or whatever. Of course, there may be a few more version-related information, etc. to make sure the server can keep track of what version files were placed into each client with the LAN sync feature and that the proper syncing will continue.

So if you have Dropbox installed on your PCs in your LAN, monitor the LAN traffic with a packet capture tool. Create/modify/delete files on one PC or the other, and you will see what I described above.

I am not claiming Dropbox's method is the best, but it uses an established protocol (RFC) to broadcast the new file/modification and to exchange application-level command/data/response and so owncloud/nextcloud should do something similar.

zephyrus00jp commented 4 years ago

A concrete algorithm would be. When a client creates/modifies (or removes) a file, and this is propagated to the owncloud/nextcloud server after the ordinary processing (transaction-wise), the server issues a notice (signed by server PKI key to avoid forgery) that this client, let's call this Client C (I am not entirely sure how the client is identified uniquely. IP address may not cut it. It is easy to spoof it later on. Maybe a MAC and other PC identification combined. If each client has its own PKI key pair, obviously, that client can be identified by its public key.) has the copy of a file, let's call it file A, and others can copy it instead of copying it from the server.

Now upon receiving the notice from the server, this client C advertises that it has the latest copy of File A by broadcasting the notice on the LAN (this is similar to UPnP). Let's pick up another client D on the LAN. Client D always listens for such an announcement on the LAN. Upon hearing this announcement,

(Possible additional step to let Client D also act as the source of the latest copy of File A.) Upon the successful completion of the file copy, client D can notify the server that it has the latest copy of the file A by sending its check sum etc. (via secure channel.)

Client C, upon hearing the broadcast, checks its file store, and knows that it already has the latest copy (I am not familiar with owncloud/nextcloud internal, but I assume it has some file ID (based on a hash?)/relative-path-to-the-repostiry-root to make it easy for the identification.) ignores it.

Other clients which may have missed the earlier broadcast from Client C (broadcast is not reliable, you see), can now talk to Client D to initiate the copy process if they don't have the latest copy of file A.

The broadcast can be repeated at 1, 2, 5, 10 min and thereafter 1 hour apart for a day or something like that.

That was a push side.

The pull side would be simply this. Currently, owncloud/nextcloud clients can begin downloading files from a server after it learns the server has newer files. Upon figuring that, instead of starting the downloading files from the server, the client can initiate the request from a LAN by broadcasting the request:

Clients that have such files will respond that they have the latest copy of certain files among the requests. What happens afterward is left as an exercise to the reader. The server can ascertain that a client, let's call client E, seems to have such files by checking such a claim by checking the hash of files sent from the client. The requesting client already knows the authentic hash of the requested file, and so at the end of copying, the mismatched check sum means either a corrupted file or bogus claim. Anyway, requesting client can pick up the positive response among such responses and begin talking for file transfer.

It is doable securely. Just fill in the details and we are all done. Famous Last Words.

biva commented 3 years ago

Thanks @zephyrus00jp ! In addition, shouldn't we verify the authenticity of the file to be downloaded? (I think NC uses this anyway to verify if 2 files are different or not, I don't remember if it's a hash or something similar).

@er-vin thanks for the enhancement label: is there any chance that the NC team considers this an interesting feature? Thanks again :)

er-vin commented 3 years ago

@er-vin thanks for the enhancement label: is there any chance that the NC team considers this an interesting feature? Thanks again :)

Interesting feature? Definitely yes. But to be honest there are a ton of other features waiting as well and time is limited. Although I like the idea, don't expect something from us any time soon on that one.

biva commented 3 years ago

:( I understand time is limited and I respect your job and your choices: you develop Nextcloud and I'm nobody to complain because of your choices as I'm just a user. But how to advocate this topic? Indeed, in my opinion, Nextcloud could consider it as a higher priority because:

  1. It is a project that would have an impact in Africa (and other low-connectivity regions), a place which is not often considered for its specificities in IT projects
  2. It would have a climate change impact, and the IT sector should, as others, do its part (and maybe more than others if we consider the growing impact of IT in the CO2 emissions)
er-vin commented 3 years ago

Well, I know... your arguments are sound. Still, I won't make empty promises and in all likelihood it'd come after more pressing matters we have.

Of course, if someone starts working on it I'd be happy to help review.

biva commented 3 years ago

Thanks @er-vin :) By curiosity, does it mean that you will bring these arguments in your internal discussions to decide if Nextcloud can make a specific effort for this issue? Thanks again for your consideration.

er-vin commented 3 years ago

It means the same that I've said before: I like it, currently we don't have the bandwidth for it, we probably won't have the bandwidth for it for a long while.

Railsimulatornet commented 3 years ago

I really hope that this feature will come, it would save a lot of traffic.

Frizou commented 3 years ago

Nextcloud.com knows LAN Synchronization functionality is missing, see: https://nextcloud.com/fr_FR/compare/

biva commented 3 years ago

Indeed, that's even the single missing functionality for Nextcloud (and to be honest, only Dropbox and Egnyte have this feature according to this table in 2018. I don't know the status today).

HammyHavoc commented 3 years ago

Would love to see this happen. Dealing with large data sets is a complete ball-ache.

1989gironimo commented 3 years ago

I'm using a split DNS at home where the hostname outsite of my LAN resolves to the public ip and inside it resolves to the LAN ip. So I'd say this feature isn't really needed, if your sync clients and your server are inside the same LAN which should be kinda obvious for LAN sync (even seperated via VLANs and similar it would work pretty much the same)

HammyHavoc commented 3 years ago

On paper, that's nice, in reality, not everybody always works in the same place, frequently we're working in the field and need to sync data with each other and not ravage our data plan. This is where things like Resilio really shine versus Nextcloud.

biva commented 3 years ago

@1989gironimo you did not understand the purpose of this issue: the point is not to sync with the server on the same LAN (which is already working by default), the point is to sync with other clients on the same LAN.

1989gironimo commented 3 years ago

My bad. (But still unsubscribed because of the "why isn't it already implemented" comments, which doesn't add any value to this issue)

mscottnelson commented 3 years ago

I’ve been considering an alternate solution to this problem, which has been experimented with before: a Nextcloud server on the local LAN that is synchronized with a Nextcloud server wherever, plus a dns forwarder so that when you are on the local LAN you reach the local server, and when you are not, you reach some other Internet-facing server.

I’ve been reading about multi-node setups with great interest, but for the average home user, even just having client -> local server <-> public server <- another client would be a pretty awesome setup for those of us with unreliable connections.

If anyone can point me in a direction to make this setup happen in 2020 or 2021, I imagine it would help others too. It’s not desktop client p2p sync (LAN or otherwise), but it would provide many of the same benefits to anyone hoping for LAN sync: the ability to have Nextcloud “just work” even while disconnected from a public-Internet-facing Nextcloud.

I saw previous comments about Galera clusters and syncthing. I’m not sure if there’s a known way to handle the inherent state problems for a transactional system in a network you expect to be slow and/or partitioned. Still, if there’s a known safe way to get this done today, that would be great to know about in the absence of built-in p2p sync.

biva commented 3 years ago

This second approach @mscottnelson is interesting indeed. To clarify our use-case:

I think that you might have more feedback if you start a dedicated issue on this.

kitatek commented 3 years ago

So much literature has been written about this request, but no code ..., that I guess there is something blocking: funding for client development, or strategic direction ?

The pros for Nextcloud:

The cons ?

As a reference, one company was doing exactly that (Groove p2p LAN friendly file sharing), and was very quickly acquired by Microsoft who killed the great product. Reasons (guessed, nothing official), is that the big ones wanted to :

  1. keep selling more servers software(Sharepoint at that time) and OS licenses
  2. keep everything under their control (nothing flowing freely on the LAN, every information through their servers)

You will see the exact same pattern of centralization to GAFAM in the chat "I.M." field: Apple migrated the brilliant LAN i-chat protocol "Bonjour" towards the Apple servers, and finally turned-off the LAN feature. Again, revenue (data, AI, advertisement, icloud space subscription) and control (reading and recording your thoughts within seconds) played in the game.

These movements are in total opposition with engineers-cherished technical performance and users-cherished convenience and common-sense. And with green IT as well, as it over-inflates the usage of internet and server load, further killing our planet.

So the question to Nextcloud team is whether this requirement conflicts with or serves their interests.

What I see that serve Nextcloud values is:

For the gozillion of would-be blockchain file system projects, just let them contribute an external storage plugin without disturbing Nextcloud team, would be my suggestion.

biva commented 3 years ago

@er-vin and Nextcloud team, did you ever consider grant funding to develop such functionality? Indeed, it meets key objectives of donors (helpful for Africa + climate compatible). There are a lot of grants available for that (look for example, but not exhaustively, https://www.ictworks.org/category/funding/).

derWalter commented 3 years ago

lets push this more!!

AndyXheli commented 3 years ago

@biva Why don't you setup an internal dns server and point your recorders internally then all your internal clients will sync with the server locally and still have the ability to sync via wan. Best practice is always to setup a internal DNS server to handle all DNS requests

biva commented 3 years ago

@andyxheli I guess you did not answer the issue here. As explained (https://github.com/nextcloud/desktop/issues/2010#issuecomment-698318677), the point is not to sync with the server on the same LAN (which is already working by default), the point is to sync with other clients on the same LAN.

adegans commented 2 years ago

I'd like Lansync too. All clients on the lan should have some auto discovery for each other similar to Bonjour. Then match files and do a sync locally.

In a perfect world there would be one master once all local devices are the same to get the server up to speed.

derWalter commented 2 years ago

@biva Why don't you setup an internal dns server and point your recorders internally then all your internal clients will sync with the server locally and still have the ability to sync via wan. Best practice is always to setup a internal DNS server to handle all DNS requests

thats is just technically right. but this should work out of the box and not require user intervention on any level, install and work ffs :D

wizdude commented 2 years ago

thats is just technically right. but this should work out of the box and not require user intervention on any level, install and work ffs :D

this isn't the problem we are trying to solve. the issue relates to clients syncing between themselves.

use case: nextcloud at central office. multiple clients at a branch office on a local LAN. all of the clients sync with the nextcloud in the central office over the WAN. we want them to sync between themselves.

i originally saw this (and used this) with btsync a really long time ago and i thought it was amazing. solves so many issues, like when setting up a new client in a remote location. the client can sync all data from others on the local LAN rather than over the WAN.

the next jump for nextcloud is to scale out for common larger scale solutions such as head office/branch office scenarios.

Railsimulatornet commented 2 years ago

Any news?

ktuulos commented 1 year ago

Just bumping this up. The local sync feature would be extremely useful. I also have several computers at home, from which one is always open and 3 laptops. My actual internet connection is pretty slow, so it would be very useful to allow laptops to sync changed files from the always-running computer in my home LAN - just like it is done in Dropbox.

ostasevych commented 1 year ago

Let me share my experience. I represent a community organisation, which offices are located in different parts of Ukraine. We are using Nextcloud to collaborate, as it provides data privacy what is crucial for our community. Since the end of Feb our infrastructure and civilians are attacked by russians, that lead to the power/network outages. So, the main server located in Ukraine cannot be secured with the reliable power anymore by placing it any part of the country. So, the need to have alternative ways to sync between clients when there's no access to the main server exists. LAN sync between clients and sync between servers using p2p are two main requests, which may solve the task.

HammyHavoc commented 1 year ago

@ostasevych Wishing you health, happiness and peace from rainy Liverpool. ❤️ 🕊️ ✌️

biva commented 1 year ago

Thanks for sharing your experience @ostasevych , and we admire your attitude in this war: good luck to you and your people.

Regarding this issue, I see that we share the same challenges as you about power and network availability in Africa... I also see that this feature is the single "red" missing feature in the comparison with other solutions (https://nextcloud.com/compare/), so I hope Nextcloud will have the bandwidth soon to consider this ✌️

wargreen commented 1 year ago

Another bump. We are a little association that promote freedom, local services and network and low-tech (as possible). We offer the services in a "pay what you want" idea. So we have not lot of money for bandwidth. Save us lot of Go and make the data the more local as possible !

zephyrus00jp commented 1 year ago

Thanks @zephyrus00jp ! In addition, shouldn't we verify the authenticity of the file to be downloaded? (I think NC uses this anyway to verify if 2 files are different or not, I don't remember if it's a hash or something similar).

@er-vin thanks for the enhancement label: is there any chance that the NC team considers this an interesting feature? Thanks again :)

It has been a while since I wished the client sync in LAN setup exists.

Yes, we need to authenticate the file integrity (either in advance, by a server declaration that this client has the authentic file with a certain time stamp and a published hashsum signed by PKI key of the server, or after the fact check by checking hash sum published by the server. The client that received a copy from another client can check the checksum.). I prefer "in advance" check to avoid DOS attack by errant clients.

Somebody has to write a detailed spec maybe based in my idea outlined in https://github.com/nextcloud/desktop/issues/2010#issuecomment-660140431

I wondered if we can create a plugin for this, and it seemed a good prototype approach, but then I realized that a server side change is unavoidable because

So without the server's publishing such data, we cannot exchange files in a trusted manner. And of course, the clients need to be changed to

  1. Normal copy/download.
    • detect the remote server change,
    • instead of requesting the modified files from the server, ask the LAN to see if the file is available from a client on the LAN. If so, request and copy.
    • If the file is unavailable in the LAN or the copy from the LAN failes, request a copy from the server.
  2. Of course, we need to receive the active notification from a client which says that a file has been uploaded to the server, meaning a fresh local copy is available on the LAN, and act accordingly.
  3. Upload to the server.
    • publish the file to the server.
    • We may want to broadcast tot the LAN that a file has been modified on a client and thus others may want to copy it EVEN BEFORE the server upload. (I am not sure of the validity of the approach. In this case, the file and copies of it will be in a special case where the upload by a single client ought to flip the files as "the copy" on the server. This may complicate matters quite a lot, and may be left untouched.)

So details have to be worked out, but it does not seem to be rocket science. Already a working example is in existence. Implementing a similar feature needs attention to the details with enough tests.

With more and more IoT-like applications done in the remote places (agricultural farms, forests, etc.), it would be great to have this local client sync since there is always the cost and bandwidth consideration, which may not be an issue at all when you are inside an office connected by fiber to other offices and data centers.

I feel really stupid if a CD image, say, 500MB, had be copied to a note PC and another note PC has to copy it from the server again using up the bandwidth. With Dropbox, the first note PC needs to copy it from a server, but the second PC copies from the first note PC via LAN. That is cool and cost-effective. You can try setting up everybody's PC with different OSes to copy files among them, and even then, people need to manually copy files. (unless you use another tool that won't interfere with owncloud/nextcloud client.) If you forget to copy a file, that file will be fetched from the remote server again using precious bandwidth and cost in a remote experimental station. Tough.

If somebody funds the effort, I may be able to implement this in the long run.

ckruetze commented 1 year ago

I wondered if we can create a plugin for this, and it seemed a good prototype approach, but then I realized that a server side change is unavoidable because we need a certified list of file(s) with their timestamp and hash values created by the server (which, I think, already exist in a format, which may need tweaking for our purposes), and we need at least a small certificate or notice that says a client A on LAN has file F, etc., which I don't think the current server does not produce. So without the server's publishing such data, we cannot exchange files in a trusted manner. And of course, the clients need to be changed to

Why? Why ask the server who has the file? Leave it to the clients to decide if they want to share a file on the LAN. That way the server needs an up to date information on who is on what network. How about if a client want's a file the client asks other clients who are local if they have file X? This could be done via a broadcast and anybody can answer to leave the server completely out of it. Or the server publishes a list of clients in the same subnet. Then a client could directly ask all other clients if they have that file.

Also if the server sends over the full folder structure with filenames and checksums to each client regularly in theory clients could LAN sync even when the server is offline. If client A knows it wants file X with checksum ABC, it could ask other, local clients if they have that file. If they do, the sync could be done locally without ever telling the server.

zephyrus00jp commented 1 year ago

I wondered if we can create a plugin for this, and it seemed a good prototype approach, but then I realized that a server side change is unavoidable because we need a certified list of file(s) with their timestamp and hash values created by the server (which, I think, already exist in a format, which may need tweaking for our purposes), and we need at least a small certificate or notice that says a client A on LAN has file F, etc., which I don't think the current server does not produce. So without the server's publishing such data, we cannot exchange files in a trusted manner. And of course, the clients need to be changed to

Why? Why ask the server who has the file?

Very simple. Are we going to trust any computer node that pops up on the LAN out of blue? I would like my PC to trust only the files from PCs that can upload files to the server. That is, my PC computer only trust other PCs who can talk to the server (with login).

As soon as my PC can know the trustworthiness of the other node, it can receive or send files (we don't want to exchange the files with random Joe on the LAN.)

In my scenario, the clients

ask the LAN to see if the file is available from a client on the LAN. If so, request and copy.

This is queried by a broadcast on the LAN. But whether the file copy is legitimate and not outdated is checked by an authorization from the server in the end.

That check can be either done by asking the file timestamp and hash sum info to the server, or by receiving the (file pathname, timestamp, cheksum, etc) which is signed by the server from the other client on the LAN which responds to the broadcast.

(Now, if the other client has a certificate signed by the server that says this client has the file with a certain timestamp and published checksum, then we can trust it to a point. If the timestamp is newer than what my PC has, the client may decide to copy from the other client without checking the server at all that it is indeed the most up-to-date copy if the connection to the server is currently down. I am assuming that on the average this is a win. The cost of local copy is much lower than the remote copy and if the chance of the copy on the LAN outdated by another upload from another remote client is low, it is certainly a win. Once the connection to the server is restored, if the copied file is outdated somehow, the copy from the server happens.)

I would rather make the security very tight at the beginning.

Somebody got to write the base straight-forward scenario and implementation specification correctly with proper security in mind and make the implementation work with tests. Then we can probably add some short-cut that may be a performance win on the average later on.

Even with a crude, not so efficient implementation in the beginning, not having to use often flaky international connection et al all the time is a big win in principle, at least it was so in the situations where I was in a few times in the past. Dropbox saved the day thanks to its LAN sync.

I was shocked when I switched to owncloud due the lack of LAN sync and seemingly too much I/O in comparison to Dropbox to check the modification of files (Once owncloud probably caused a few times more I/O than Dropbox. I compared the operation on two PCs next to each other on the office LAN. I think owncloud became relatively better, but have not measured it in detail since I have stopped using Dropbox several years ago. I have not measured I/O performance of nextcloud in this regard. It would be instructive to do so if somebody has an active Dropbox account.)

mscottnelson commented 1 year ago

If the sync is thought of as p2p in the first place, then the authoritative server effectively becomes a Certificate Authority. As long as your peer has a valid key signed by your CA I’m not sure why you couldn’t trust them to at least be who they say they are, which can prompt a user intervention to allow/deny sync with a “new” peer that was at some point see-able to your central authority.

NextCloud could maybe wrap the source of eg syncthing: https://github.com/syncthing/syncthing

zephyrus00jp commented 1 year ago

If the sync is thought of as p2p in the first place, then the authoritative server effectively becomes a Certificate Authority. As long as your peer has a valid key signed by your CA I’m not sure why you couldn’t trust them to at least be who they say they are, which can prompt a user intervention to allow/deny sync with a “new” peer that was at some point see-able to your central authority.

You have a point there, but I don't want the central server to become the full CA. Isn't this too much for NextCloud (or OwnCloud) for that matter. We may leave it to somewhere else.

My point is that client only needs to have the central server key and does not have to have the keys of OTHER clients at all. There is a problem of identifier of the each client.

[When the WAN connection is available], So when some client says, I have new updated file(s) and broadcast the information on the LAN, I would like my client to verify that the broadcast information is legitimate Because there is no reliable way to identify a client uniquely, a different client can simply copy such broadcast and reuse it (REPLAY). [Typos fixed] I would like my client to ask the other client securely without the needs for PKI infrastructure (I am thinking of DH-based protocol) with a NONCE (a random number generated now) and insist that the other client has the data of the list of files (pathname, checksum, updated time)+ NONCE signed by the central server. When the other client returns the signed such data with correct NONCE, then I can trust the other client indeed can talk to the server and did not simply copy someone else's broadcast. Only then the local copy would start.

Yes, this does not work without the WAN connection to the central server. But I think this covers 95% of the use cases I have in mind. The amount of network bandwidth used is small compared with the real duplicate copying one sees.

The security model and attack model when the WAN connection is not available need to be studied carefully.

NextCloud could maybe wrap the source of eg syncthing: https://github.com/syncthing/syncthing Someone in the know can comment on this.

To me, get the security model correct and algorithm correct is the first step. Once that is done, coding follows naturally (in principle. Famous last words. :-)

Edit: Typos fixed.

jonvog commented 1 year ago

I just wanted to leave a quick +1 on this. There's 2 reasons I still have a dropbox subscription: it handles bigger files in general way better than NC (at least with my particular setup), with bigger files I mean multiple GB files. And the LAN sync thing. I work with film and I often deal with files north of 10GB that need to be synced from my main workstation to my laptop. The scenario: I've been working on a film for a couple of days, have several cut versions, each 10-20GB. Then I need to travel for one day and need everything on my laptop. With Dropbox I just fire up both systems, let it run a couple of minutes and I'm good to go. With NC and my 50mbit internet connection, I have to plan for a couple of hours of sync time, before being able to take my laptop out of the studio, ready with all the files. Like, I'm not a dev and I don't know the roadmap and how complex it would be to implement, but I just can't see how this isn't a central thing on the list? Please, save me and others those hundreds of dollars every year and the hassle of having to have two cloud solutions. (And the waste of ressources in terms of sustainability is a super important reason too!!)

kitatek commented 1 year ago

The solution here by @zephyrus00jp is probably the best to be implemented now for these reasons:

pylorak commented 8 months ago

I know this has been discussed already, and I know this is an unpopular solution in the eyes of many because LAN sync should just work out of the box. And I agree with that, LAN sync should just work.

Still, until it is implemented, I do need to point out that an internal DNS server is often an optimal solution if you want LAN sync now:

The latter two requirements (a server and linux knowledge) would be the two show-stoppers for most people under normal circumstances. However, given that LAN sync only makes sense if you have a local NC installation, basically all private users who want LAN sync already satisfy these requirements.

This still leaves corporate users as problematic though, as they cannot do the necessary changes themselves in the network. It is not easy to get such changes implemented by the admin in a corporate network, even if the roadblocks are most times not of technical nature. One more reason why NC should finally implement LAN sync.

HammyHavoc commented 8 months ago

I know this has been discussed already, and I know this is an unpopular solution in the eyes of many because LAN sync should just work out of the box. And I agree with that, LAN sync should just work.

Still, until it is implemented, I do need to point out that an internal DNS server is often an optimal solution if you want LAN sync now:

  • It works
  • It is secure
  • It is actually easy to setup, as long as you already have a server and basic linux knowledge.

The latter two requirements (a server and linux knowledge) would be the two show-stoppers for most people under normal circumstances. However, given that LAN sync only makes sense if you have a local NC installation, basically all private users who want LAN sync already satisfy these requirements.

This still leaves corporate users as problematic though, as they cannot do the necessary changes themselves in the network. It is not easy to get such changes implemented by the admin in a corporate network, even if the roadblocks are most times not of technical nature. One more reason why NC should finally implement LAN sync.

Some ISP-provided modem-router hardware prevent users from doing this as the ISP forces the use of their hardware. NAT reflection/loopback et al tend to be highly lacking on a lot of consumer hardware.

Also consider that a lot of users cannot access their IPTV if they ditch their ISP-provided modem-router hardware. For some contracts, it's actually going against the agreement to use anything but what they provide in that regard.

pylorak commented 8 months ago

Some ISP-provided modem-router hardware prevent users from doing this as the ISP forces the use of their hardware. NAT reflection/loopback et al tend to be highly lacking on a lot of consumer hardware.

Also consider that a lot of users cannot access their IPTV if they ditch their ISP-provided modem-router hardware. For some contracts, it's actually going against the agreement to use anything but what they provide in that regard.

NAT reflection/loopback is a totally separate feature/function which is not required for the solution that I and many others have described (redirecting to the LAN address using a local DNS server). So the lack of this feature in consumer hardware is not a problem and is irrelevant here.

The only router feature that is beneficial for the proposed solution is the ability to configure the DNS server addresses that are handed out to DHCP-clients. This is supported by almost every router, even by ISP-provided ones. Also, even this feature is optional, because in the extremely rare case where configuring the DNS server addresses is not possible in the router, you can still configure the DNS server address statically on clients, or adjust their hosts files.

HammyHavoc commented 8 months ago

Some ISP-provided modem-router hardware prevent users from doing this as the ISP forces the use of their hardware. NAT reflection/loopback et al tend to be highly lacking on a lot of consumer hardware. Also consider that a lot of users cannot access their IPTV if they ditch their ISP-provided modem-router hardware. For some contracts, it's actually going against the agreement to use anything but what they provide in that regard.

NAT reflection/loopback is a totally separate feature/function which is not required for the solution that I and many others have described (redirecting to the LAN address using a local DNS server). So the lack of this feature in consumer hardware is not a problem and is irrelevant here.

The only router feature that is beneficial for the proposed solution is the ability to configure the DNS server addresses that are handed out to DHCP-clients. This is supported by almost every router, even in ISP-provided ones. Also, even this feature is optional, because in the extremely rare case where this is not possible, you can still configure the DNS server address statically on clients, or adjust the hosts files.

Scenario: you run a Nextcloud server for your family members. It's a holiday. They all come over to your house, they're snapping away with photos and video, sharing them with each other, and they bring your WAN to its knees because they haven't got the statically configured settings. Is it fringe? Sure.

pylorak commented 8 months ago

Some ISP-provided modem-router hardware prevent users from doing this as the ISP forces the use of their hardware. NAT reflection/loopback et al tend to be highly lacking on a lot of consumer hardware. Also consider that a lot of users cannot access their IPTV if they ditch their ISP-provided modem-router hardware. For some contracts, it's actually going against the agreement to use anything but what they provide in that regard.

NAT reflection/loopback is a totally separate feature/function which is not required for the solution that I and many others have described (redirecting to the LAN address using a local DNS server). So the lack of this feature in consumer hardware is not a problem and is irrelevant here. The only router feature that is beneficial for the proposed solution is the ability to configure the DNS server addresses that are handed out to DHCP-clients. This is supported by almost every router, even in ISP-provided ones. Also, even this feature is optional, because in the extremely rare case where this is not possible, you can still configure the DNS server address statically on clients, or adjust the hosts files.

Scenario: you run a Nextcloud server for your family members. It's a holiday. They all come over to your house, they're snapping away with photos and video, sharing them with each other, and they bring your WAN to its knees because they haven't got the statically configured settings. Is it fringe? Sure.

Static client configuration was a last resort in my answer. Realistically, you can configure this in almost every router, including ISP-provided ones.

Also, what are you arguing about? I didn't say native support for LAN sync in NC isn't needed, I actually said the opposite of that. I was just pointing people to a workaround in the meantime that will work for most private users of NC.

HammyHavoc commented 8 months ago

Some ISP-provided modem-router hardware prevent users from doing this as the ISP forces the use of their hardware. NAT reflection/loopback et al tend to be highly lacking on a lot of consumer hardware. Also consider that a lot of users cannot access their IPTV if they ditch their ISP-provided modem-router hardware. For some contracts, it's actually going against the agreement to use anything but what they provide in that regard.

NAT reflection/loopback is a totally separate feature/function which is not required for the solution that I and many others have described (redirecting to the LAN address using a local DNS server). So the lack of this feature in consumer hardware is not a problem and is irrelevant here. The only router feature that is beneficial for the proposed solution is the ability to configure the DNS server addresses that are handed out to DHCP-clients. This is supported by almost every router, even in ISP-provided ones. Also, even this feature is optional, because in the extremely rare case where this is not possible, you can still configure the DNS server address statically on clients, or adjust the hosts files.

Scenario: you run a Nextcloud server for your family members. It's a holiday. They all come over to your house, they're snapping away with photos and video, sharing them with each other, and they bring your WAN to its knees because they haven't got the statically configured settings. Is it fringe? Sure.

Static client configuration was a last resort in my answer. Realistically, you can configure this in almost every router, including ISP-provided ones.

Also, what are you arguing about? I didn't say native support for LAN sync in NC isn't needed, I actually said the opposite of that. I was just pointing users to a workaround in the meantime that will work for 99% of all private users of NC.

I'm not "arguing" about anything, and whether you support LAN sync or not doesn't negate the suggestions made, which aren't appropriate for a sizeable number of people.

E.g., https://www.quora.com/Why-dont-ISP-routers-allow-you-to-change-the-DNS/answer/Khairul-Aizat-28

On some ISP-provided hardware, you must use the ISP-provided DNS. This affects what gets dished out via DHCP in terms of DNS.

sunjam commented 8 months ago

NextCloud could maybe wrap the source of eg syncthing: https://github.com/syncthing/syncthing

Was already discussed and formally rejected here:

Do note that Syncthing can already be integrated with Nextcloud folder mounts as described here: https://github.com/nextcloud/server/issues/8384#issuecomment-368673597 Also see the discussion on adding delta sync support: