search

uktrade / mobius3

Continuously sync folder to S3, using inotify under the hood
MIT License
50 stars 3 forks source link

changes made during the system offline is not getting sync #45

Open vinayKs21 opened 3 years ago

vinayKs21 commented 3 years ago

When my system is not connected to internet or when I just not run the program and make changes to the folder which I have sync with my S3Bucket, and when I connect to internet or run the program, mobius3 don't sync the new changes of the local folder to the S3Bucket instead what it do, It just overwrite my local folder changes with the data available in my S3Bucket. Is this a bug or this feature is not available. Please help.

michalc commented 1 year ago

Hi @vinayKs21,

Apologies for the extremely slow reply, and thanks for the report - is this still a problem?

Thanks,

Michal

jgm-ktg commented 1 year ago

I have also seen this problem. Upon starting mobius3 local data is clobbered and replaced with what is in the bucket. There is also some bug where mobius3 will stop syncing but also not exit. This allows local changes to accumulate for some time before it is noticed and mobius3 is manually restarted. This makes it particularly painful when local changes are clobbered.

jgm-ktg commented 1 year ago

@michalc Is mobius3 likely to get any attention anytime soon? Are there any other inotify-based sync tools in the spirit of a "dropbox client" for s3? The idea of mobius3 fills a big need but the implementation doesn't seem robust yet. Thanks

michalc commented 1 year ago

@jgm-ktg It does get some attention, and I have thought a bit about this issue. It sounds like the use case at our end is a bit different to yours. In ours, the source of truth is very much S3 - and its even desirable to remove files that might be present when starting up that aren't in S3.

That's not saying that we couldn't make improvements to robustness, or make it so files present on startup are uploaded. I think especially the case of mobius3 running but not being able to communicate with S3 is a case we probably want to improve. I think some improvements have been made in this respect around November by using a slightly more usual HTTP client, but suspect there is more that could be done.

But in terms of how soon further improvements will be make, I don't think I can answer that with any certainty. I think best to assume it will be a while.

Although on

There is also some bug where mobius3 will stop syncing but also not exit

If this is still a problem on the latest version, and you manage to make a reproducible example, that would make it much more likely that any issue will be fixed soon.

And finally on

Are there any other inotify-based sync tools in the spirit of a "dropbox client" for s3?

Alas I don't know of any - and in fact, I would be interested to find them too