stephenh / mirror

A tool for real-time, two-way sync for remote (e.g. desktop/laptop) development
Apache License 2.0
391 stars 37 forks source link

not all files are synched #22

Closed fezzzza closed 5 years ago

fezzzza commented 5 years ago

I have 4 subdirectories from my nominated root containing different branches of essentially the same code, each holding apx 307MiB. I call them dev, edge, offline and stable. The largest and most unchanging folder, "scripts", containing thousands of directories (thanks, composer!), has been excluded from the sync (apx 210MiB) using --exclude to keep the load down, and there is a 76MiB .git folder, leaving 21MB in 147 paths(according to the log) to sync.

On running the "mirror client" command, edge and offline will synchronise and immediately exchange files, whereas dev and stable are not touched. Similarly, updating files within edge and offline will trigger uploads, whereas dev and stable don't seem to be monitored.

In troubleshooting, I thought maybe dev and stable are reserved words, so I stopped the mirror client, renamed dev to dv on the server and client, and restarted the mirror client. No change. Then I renamed dv to dev on the client, and that folder was synched to the server, but not all the files. Stop and start the client again, and no synchronisation occurs either as an initial sync or when the filesystem is updated.

I am running linux mint 19(~ubuntu 18 bionic) without watchman installed (see #20).

This is the command line I am running: mirror client --debug-all --exclude dev/ui --exclude edge/ui --exclude stable/ui/ --exclude offline/ui --exclude dev/log/ --exclude edge/log/ --exclude stable/log/ --exclude offline/log/ --exclude dev/scripts/ --exclude edge/scripts/ --exclude stable/scripts/ --exclude offline/scripts/ --exclude dev/bin --exclude edge/bin --exclude offline/bin --exclude stable/bin/ -h localhost -l /var/www/html -r /var/www/html &

Incidentally, /stable/ui/ is a large directory I don't need to sync, and /dev/ui etc is a symlink to that directory. Similarly with */bin .

I have checked file and directory ownership and permissions, which all seem uniform throughout the nominated tree. All files in the tree (with the exception of a few that need to be owned by www-data) belong to the user running the mirror client executable, with permission level of 644.

Similarly, on the server mirror is executed by root and the files are owned by root with 644.

I have run the client with --debug-all and there is no mention of "dev" or "stable" in the log, so it seems as though mirror is not even considering these directories.

I have plenty of RAM on the client (8GB) but very little on the server (<1GB RAM + <1GB swap). Synchronising without these exceptions resulted in java memory errors on the server, but limiting the directories as mentioned has eradicated these memory errors.

I have tried moving the large trees "scripts" and ".git" away, stopped and started the client but still no sync of dev and stable, so the two large directories seem to be a red herring.

fezzzza commented 5 years ago

This seems not to be the case with v1.2.1 and watchman installed, though I haven't done extensive testing yet.

stephenh commented 5 years ago

Planning on "fixing" via #25 .