Deleting Empty Directories

[WayneSherman]

I haven't tested rclonesync yet, but I suspect this behavior will not work for many use cases: "Any empty directories after the sync are deleted on both the Local and Remote filesystems."

In my opinion, a sync and clone tool should not take it upon itself to modify or delete any files or folders in a file tree unless by explicit user request. (e.g. the user deletes a local file, so the sync tool deletes the remote is good behavior, or the user explicitly configures the sync tool to delete empty folders is good behavior)

We use a structure of empty directories as a project folder template. The project folder template gets copied to each new project folder and the empty directories are placeholders where future project files will be created.

[cjnaz]

I buy that. rclone itself works on files, and its sync doesn't specifically deal with directories which may have been removed. This will take a bit of code/logic to recognize when a previous directory was deleted on one side or the other.

[cjnaz]

Proposal:

Run directory sync after file sync (existing functionality) Directory sync:

• Track directories on both local and remote
• If deleted, delete on other size (must obviously already be empty)
• If new, and not on other side, then mkdir (if the new folder has files they it would be created by the file sync)

Filters apply. If a folder is new on one side, containing only excluded (filter '-') files an empty folder will be created on the other side.

Comments please...

[WayneSherman]

rclone support may be coming: https://github.com/ncw/rclone/issues/1837 https://github.com/ncw/rclone/issues/2404

[MuerteInHell]

Agree with your proposal for directory sync, the logic seems good.

Empty folders being deleted makes me unable to use rclonesync-v2 and rclone in production.

Alternatively, an option to not delete empty folders on either local side / remote side could also help.

[WayneSherman]

Update: I tested a recent rclone beta from a few days ago and rclone by itself worked properly with Google Drive for my test. Source was the remote Google Drive and destination was a local file system. Empty folders from Google Drive copied correctly to the local file system.

[cjnaz]

@MuerteInHell - Ok, the directory handling proposal is the plan. In the interim you can just comment out the rmdir lines from 448 to 454.

@WayneSherman - Thanks for the rclone update. The rclone enhancements don't solve the general problem of bi-directional sync. rclone sync makes the destination look like the source, but does not comprehend changes made on both sides - be they files or directories. The rclone changes will be useful in rclonesync: rclonesync needs only to make Path1 correct and then the rclone sync will make Path2 match.

[r2t2]

I want to second @MuerteInHell 's comments. Empty folders need to be left alone. As you mentioned, I'll check out the source files.

[cjnaz]

For the near term, I'll add a --remove-empty-directories switch, which defaults to off/false. If this switch is set then the rmdirs will be executed, matching rclonesync's current behavior. This is a change in the default behavior.

Doing empty directory handling properly is a bit more complex, as noted above.

Understand that the net effect will be that the empty directory state on Path1 always wins. This is because the final synchonization step is to run an rclone sync, which will make Path2 look like Path1, including empty directories on Path1.

In practice, if you delete a directory on Path2 all the files will be deleted also on Path1 but the final rclone sync will put the empty directory back on Path2. Similarly, if you add an empty directory on Path2 the final rclone sync will delete it if it does not exist on Path1.

I understand that this is a somewhat messy solution. Longer term I plan to incorporate rclonesync / bi-directional sync directly within rclone, and not perfect this issue in this script.

[cjnaz]

Changed default behavior to NOT delete empty directories after the sync, and added the --remove-empty-directories switch in V2.4.
NOTE that this is NOT a complete and proper solution for the empty directory sync issue, but its better. If you want to delete a directory, you must delete it on Path1 and it will be deleted on Path2.