Client synchronizes on each file change - grace period option wanted.

[ftiede]

The desktop client on (at least) Linux (haven't tested on other OS's) synchronizes each time a file is changed on local disk or remotely.

While I see reasoning behind this behavior, esp. in environments with many users sharing the same files, this can drag on bandwidth and/or create many versions stored on cloud-storage with a lot of small changes.

So my wish is to be able to set some kind of grace period after a local change before the new version is uploaded, possibly with a "Sync now" button.

If I can find time (and sufficient understanding of the code) I'd be willing to work on such a feature.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/27268845-client-synchronizes-on-each-file-change-grace-period-option-wanted?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github).

[MadNachos]

I could also use a feature like this....something like 'sync when files have not been touched/updated in X minutes' would really be a great benefit for our shops use of ownCloud due to the volume of temporary files generated by the software they use. The temp files do not have names that can be anticipated and often the final files share the same name as a previously existing temp file so they can't be ignored using the current options. This option would solve the only outstanding issue we have and would probably be very handy to quite a few people. Maybe. I certainly need it ;)

[ftiede]

My proposal would be to implement an option which allows to select a grace period for local2remote uploads only from 0 to 900 seconds (15 minutes) max. after the last file change was detected. It wouldn't affect downloads to accomodate for remote changes as soon as they happen, so editors don't run into too many synchronization conflicts by waiting too long.

Additionally I would implement a button (and systray icon context menu option) to kick off pending uploads immediately in case an editor decides his edits are complete and should not wait for grace period to run out.

[phil-davis]

I would appreciate this also. At the moment, if you are on a slow link and doing lots of edits, you can Pause Sync and then Resume Sync when you are finished messing with the local client-side files. But of course that means that during that time you do not receive any server-side changes. The hard part is how to design the UI so it is not intrusive/confusing to new users, does not clutter up the UI when it is a rarely used thing, but can also be found by users when they want it.

[ftiede]

I would add a section in the network tab of the client and default it to "off" (upload instantly after filechange detected, as is the current behavior). For the "instant upload" I would add an entry to the acount's context menu. Whereas I can see a clash of concept here, as the setting is probably global, while the instant sync is then per-account. Any input on this? What would be preferred? Global setting -> Global instant sync, Global setting -> per-account instant sync, or all per-account (which would make the UI admittedly complex).

I do know about the sync pause/resume option but dislike it for precisely the reason you named.

[moscicki]

It's interesting how contradicting the requirements are: we recently got a request from some of our users that change propagation between two sync clients should have less time lag ("dropbox syncs almost instantaneously").

@MadNachos: are there general public applications which have this behaviour for temporary files with unpredictable names? Or does it apply mainly to your very particular applications/environment?

@ftiede: what exactly is your concern with the number of version files or bandwitdth? If you suppose that delta sync was in place (syncing just the differences), would it still be an issue for you? Or are there other concerns?

I think answering these questions would help it better understand and prioritise your request by owncloud developers.

[MadNachos]

The sync issues we have run into seems to come down to the various applications that run engineering simulations for hours or even days while generating lots temporary files. Most are very small but some are 10-50MB. I have all of the predictable temp file names ignored but ignoring over 60% of them is not an option, many of the temp files use naming conventions that match files containing simulation resukts and other data. I believe that the software in use is simply behaving badly by storing temp files in the same dirs as the project data but the two companies that own the market can't be bothered to add the option to specify locations for temp files. The kicker is its priced like they should listen...yea, right.

Anyway, the issues we have might happen on rare occasion with more common applications and we are just hitting the one in a million chance much more often but we hit it a few times a week. I am working on a few solutions to work around this but a configurable delay between a files last modification time and syncing would be ideal. These guys don't care about instant syncing, its really just a way to keep a mirror of a days work until the nightly backup runs, so minutes of delay is a non issue. I am guessing that it's not a simple feature to add though, each sync client using the account would need to use a delay time that the server enforced or things could get ugly.

And FWIW, I think a feature like this is only needed as a bandaid for broken users and software, but those are both pretty common in a few industries and nearly impossible to fix.

[ftiede]

@moscicki: Versions of files, entries in the database, bandwidth, yes. It may be that these concerns come from my usual git workflow - work, save, work, save, check, commit.

Regarding "instant syncs": If the feature is turned off, it should not interfere with the sync as fast as the client's capable.

As for priorities: Unless I get a definitive red light, I offer to work on this myself.

[dragotin]

Hm.. This doesn't sound like a good solution to a common problem I think. I found two usecases that we try to solve here:

a) too many not easy to catch temp files. Well that is a problem, but not of the client but of the other software. I know, bad argument ;-) But still, maybe there is an option to let the software not run in a synced folder, but only automatically copy the final result files to a synced directory? That would be a clean solution.

b) permanently edited files cause too much versions and/or uploads. I mean, isn't that exactly the purpose of syncing to get an edit of another user as fast as possible? I thought so. If not, I would think that a per-file based solution is much better: A user could set a file in being-edited mode, which ignores it from syncing for now. Once the modification time does not change any more for ...15 minutes, the file is automatically synced again. For some applications, such as MS Office, it might be possible to detect automatically if they are opened for editing.

[guruz]

IMHO The better fix is to make the discovery cheaper which will be handled in https://github.com/owncloud/client/issues/2297

[ftiede]

@dragotin: wrt. b) MS-Office does create temporary files in the original file's directory while the file is actually open. So does vi (.<filename>.swp). But f.ex. kwrite/kate (KDE editors) do not create those files unless actual changes happen to the in-memory-copy. Alas, they disappear as soon as the user saves his work inside those editors, which makes this way unsuitable to detect "being-edited".

You are right, the point is to propagate changes to other users as fast as possible, which is why I suggested an option to entirely disable and keep the current "synchronize as fast as possible" behavior.

[chrsch]

Just another pov.. From my experience it is important to sync as fast as possible. I often need to explain why changes (though federated share) need to propagate so long... It even causes conflicts, because shared files not yet updated are edited.... So yes, please even faster syncing!

[guruz]

@chrsch Improving federated sharing performance is on the way

FYI @icewind1991 @PVince81

[jvcdk]

Hi there :)

I also would like to request this feature... Originally, I had thought of having a signal file in the root dir (of the synced folder). E.g. .sync.pause - if that was detected by the client, pause would automatically be suspended.

My use case: a) When I edit photos (with Shotwell) it makes multiple changes to the database (written to disk, stored in my ownCloud folder) and changes to photo files. I have once had a corruption of the database where I had to fetch an old version, and have had image file conflicts. I have not investigated the details of how they happen - but for now I resolve to pause sync manually.

b) In my personal project development folders, which I have also added to ownCloud, also sees the same problem, and I have also seen files corruption here. Yes I know, I should probably pull them out of ownCloud and set up a git account on my server. But I haven't... yet.

Anyway... So reading this my thought was a) You could pause the sync by "touching" a file .sync.pause in the root directory of the ownCloud share. b) After 15 minutes of inactivity, the file would automatically be removed and sync commenced. c) If file was removed manually, sync would also be commenced immediately.

This would allow (at least Linux users) to make a special command or a wrapper for known critical applications to add/remove .sync.pause as needed.

@ftiede How far did you get with implementing this? I was actually also thinking if I could find a few hours to put into this.

@dragotin Any news on what is the priority or position of this from the Owncloud team? I am asking you, since you are the only one with the ownCloud memeber tag.

BR Jørn

[hodyroff]

Could be made configurable yes. We would take such a feature from the Community but not a priority right now.

[Ninister]

was there an progress in the last years? I would also request this feature. I got problems with word and CAD-Software. My client runs on a Mac mini, that I use as a server(just data storage) with file sharing in a local network. Sometimes I want to save the changes in Word, but have an error and word re-opens the unedited document, all changes get lost. I often copy invoices from one folder to another and then just edit with new data, but after all work I can't safe the original document. I don't exactly know what's the problem, but its definitely the combination of word and owncloud. The other problem with cad-files is like @ftiede and @MadNachos already described. its just nonsense(for me) to upload files with 80-120 MB over and over in short periods. this blocks other uploads of smaller files.

[hodyroff]

With Windows 10 VFS - while MS Office has locked/opened a file it is not saved. It is saved after it is closed and not edited anymore. Can't follow the steps of the issue you describe. We are trying to mimick the behaviour of the application, if the application locks the file, we don't sync it to ownCloud, when its unlocked, we do.

For Linux and MacOS this is likey different as the same type of lock doesn't exist in either. But only if the application saves, we can sync.

[q2apro]

Hello from 2024.

When you work with big files, e.g. Photoshop files of 100 MB, you usually save your work every 30 seconds or so.

With each save Nextcloud starts to synchronize the file.

So after 30 min work, I got a used bandwidth of 6 GB.

⚠️ An option to delay sync in the desktop client is a MUST (!)

[TheOneRing]

Hello from 2024.

When you work with big files, e.g. Photoshop files of 100 MB, you usually save your work every 30 seconds or so.

With each save Nextcloud starts to synchronize the file.

So after 30 min work, I got a used bandwidth of 6 GB.

⚠️ An option to delay sync in the desktop client is a MUST (!)

This is the ownCloud repo

[Tsunayoshi]

i support this request. i got lately big trouble with a slow connection on server side (forced to vpn) which lead into blocked files on the client side, that are changed constantly in few seconds. this then lead into crashed and errors in the programs that use these files. i would love a grace period i can define for my client. Just to make clear: this is not office related, its normal text files... right now i have only 2 options: close the client, which stops all sync or blacklist the files that are constantly changed. both lead into a possible data loss for me.

[Tsunayoshi]

For ppl who dont have 10years to wait for a solution. exclude the file from syning (ignored files) make a windows/cronjob task that copy the file every x minutes to a different folder/name. make sure this file is not excluded.