Suggestion: Change recommendation for cron to systemd

[3ter]

Hi again,

I wondered why you were recommending cron for this task. I find it could easily lead to a conflicting state if the repeating window isn't small enough (and then there's a lot of unnecessary syncs).

I'm mainly curious to know your reasons so feel free to close this issue.

I've tried to get this right using systemd executing a script on startup that first triggers a sync and afterwards syncs on changes monitored with inotifywait.

rclonesync_watch.service

``` # /home/3ter/.config/systemd/user/rclone-sync-gdrive.service [Unit] Description=Sync Googledrive with the local folder on startup and on change afterwards [Service] ExecStart=/usr/bin/perl /home/3ter/scripts/rclonesync_watch.pl [Install] WantedBy=default.target ```

rclonesync_watch.pl

```perl #!/bin/perl use strict; use warnings; use v5.30; our $local_dir = '/home/3ter/googledrive'; our $remote_dir = 'googledrive:/'; our $rclonesync_exe = '/home/3ter/rclonesync/rclonesync.py'; our $inotify_opts = "--event " . join(' --event ', qw(modify attrib close_write moved_to create delete)) . " --recursive $local_dir"; sub wait_until_file_silent { while (1) { my $rc = system( "inotifywait --timeout 5 $inotify_opts" ); # Return code 0 shows us that there was another event triggered return if $rc; } } sub watch_folder { if (system( "inotifywait $inotify_opts" ) == 0) { wait_until_file_silent(); say qx( python3 $rclonesync_exe --verbose $local_dir $remote_dir 2>&1 ); } return; } say '[rclone] Watching locally.'; if (-d $local_dir) { say 'Sync on startup'; say qx( python3 $rclonesync_exe --verbose $local_dir $remote_dir 2>&1 ); while (1) { watch_folder(); } } else { say "$local_dir isn't a valid directory!"; } ```

[cjnaz]

Interesting. Why didn't I go this route? Ignorance. I'll do some digging on inotify, as I've not used it before.

lead to a conflicting state if the repeating window isn't small enough

This is the role of the lockfile - it prevents concurrent runs on the same targets.

Questions

• Can inotify detect changes on a remote filesystem, such as Google Drive? If not, then do I still need the periodic sync method for pulling remote changes?
• Is there a reason to not incorporate the inotifywait functionality into rclonesync itself?
• I'm unfamiliar with running unit services files from systemd as a non-root user (I'm on Centos 7.8). Can this unit file be triggered on boot or must the user log in and then started from the login scripts?

[cjnaz]

From https://www.linuxjournal.com/content/linux-filesystem-events-inotify:

Recalcitrant users should be confined to Ultrix on a VAX until they develop sufficient appreciation for modern tools and approaches, which should result in more efficient Linux systems and happier administrators.

Sweet.

[3ter]

lead to a conflicting state if the repeating window isn't small enough

This is the role of the lockfile - it prevents concurrent runs on the same targets.

I meant that if you have cronrunning, change the file, shutdown (croncouldn't sync before the shutdown) and then change the file on another device. The same applies in the time after startup when you haven't fetched the potential changes from the remote.

• Can inotify detect changes on a remote filesystem, such as Google Drive? If not, then do I still need the periodic sync method for pulling remote changes?

It seems grive2 is doing it somehow so it should be possible. EDIT: I was mistaken here. They also use a timer, but the window is pretty small with 5 seconds. So that's an open question for them, too.

Timer unit file

``` [Unit] Description=Google drive sync (fixed intervals) [Timer] OnCalendar=*:0/5 OnBootSec=3min OnUnitActiveSec=5min Unit=grive-timer@%i.service [Install] WantedBy=timers.target ```

• Is there a reason to not incorporate the inotifywait functionality into rclonesync itself?

I've asked that myself. I think it would be the right place for it to make rclonesync easy and efficient to set up.

• I'm unfamiliar with running unit services files from systemd as a non-root user (I'm on Centos 7.8). Can this unit file be triggered on boot or must the user log in and then started from the login scripts?

You'd want to enable lingering for that purpose, so yes.

[cjnaz]

Musings and Q...
Some services like Drive and Dropbox have local agent apps that monitor for changes on the local side, and their local agent is probably told of changes occurring on the remote/cloud side. If their API for getting notified of cloud-side changes is published/public then a local agent could be implemented (or just use the cloud provider's agent app). One advantage could be a unified local agent, like rclone itself provides a unified cloud access tool.

One solution with rclonesync could be a CLI switch to tell rclonesync to enter a monitoring loop, which would only be appropriate if rclonesync has inotify visibility on both paths.

However, if both sides have inotify support, would it be a better solution to just make the changes on the other side directly, rather than mess with the rclonesync process flow (list everything, look for diffs, make corresponding changes)?

Reading up on rclone mount, I see this note:

systemd

When running rclone mount as a systemd service, it is possible to use Type=notify. In this case the service will enter the started state after the mountpoint has been successfully set up. Units having the rclone mount service specified as a requirement will see all files and folders immediately in this mode.

[cjnaz]

Related... https://github.com/rclone/rclone/issues/4152

[cjnaz]

Closing this for now. My take is that rclonesync isn't the right tool to act as a local agent and be notified and react to changes as they happen. If we had an agent it could take care of individual files that change, and not have to scan the sync tree at all.