Enhancement: Tray Status for Syncing Remote Machine

First of all, great work on this! I use it across all machines on our tiny home network.

When syncing between two Windows 10 Pro x64 machines and a Windows 7 Enterprise x64 machine, only the machine receiving the data indicates "Synchronizing" status in the tray icon. I would enjoy an option to either 1) consider outbound/remote syncing as status "Synchronizing" or 2) a separate "Remote Sync" status with it own configurable color.

Thank you.

I believe how I implemented the status icons is in accordance to the folder status in Syncthing's web UI so I would prefer to keep the meaning of "Synchronizing". That speaks for option 2 (inroducing a separate status) which would be shown if at least one remote device is synchronizing.

However, I'm still not sure how useful that would be in practise because one can not easily distinguish whether the remote device is actually syncing or whether it is in an error state (and therefore not completely in sync yet). E.g. my local Syncthing instance currently shows one remote device which is persistently "synchronizing" but in reality it is actually stuck at 99 % with one file missing. It won't progress anytime soon until I'm going to fix the issue on the remote device which prevents it to sync this file. So the "Remote sync" status would actually lie in this case as there's no further syncing happening at this point. The remote is actually in an error state.

I'm also wondering about the perceedence this new overall status would have. So far "Synchronizing" has preceedence over "Out of sync" which has perceedence over "Scanning" which has preceedence over "Idle". Not sure where it would fit in best.

Maybe it makes more sense to add more information to the tooltip? It currently shows "Connected to $devide_name". I suppose it wouldn't hurt to add the device status there.

In terms of the precedence of the different statuses, I would think that both "pulling changes from remote" and "pushing changes to remote" would count as "Synchronizing", because in both of those cases my local Syncthing instance is actively transferring data in order to get local and remote in sync... which is the opposite end of the scale from "Idle". If you disagree with categorizing that as "Synchronizing", I would say that the "Pushing" status should fall between "Synchronizing" and "Out of Sync", because it—like "Synchronizing"—is an "actively transferring data" state. Either way, I definitely want to weigh in with a +1 on the suggestion by @meztup that the status icon would be much more useful if it changed to some sort of "busy" indicator when Syncthing is sending or receiving data.

As for your one stuck file: to my mind, a constant "Syncing"... "Syncing"... "Syncing"... indicator is actually a perfectly appropriate way for the tray icon to represent that error condition. (Provided that there is no way to distinguish the error from normal syncing behaviour, of course. If you could distinguish, then something like an amber exclamation-point overlay on the icon would be the ideal way to indicate an error. But I digress). As you say, your local Syncthing instance shows that remote as persistently "Synchronizing"... because it is, in fact, in an active state of "trying to sync" over and over again endlessly. Once you fix the issue on your remote, then your Syncthing instance will be in sync, at which point it would make sense to show a static green "Idle" indicator. Until then, though, shouldn't the Syncthingtray icon accurately reflect the state of your Syncthing instance, instead of concealing the problem behind a spurious "everything is in sync and idle" indication?

As it stands right now, when I drop several gigs of music into the sync folder on my desktop system, it takes some time to push them all to my remote server. During that whole time, the Syncthingtray icon is sitting in a stationary green "idle" state. To my mind, that's misleading and kind of defeats the purpose of having a tray icon at all. I would strongly expect that while this process is going on, the tray icon would show some sort of "Busy" state, whether that's a different colour or a twirling animation or whatever. That way, I could keep doing other things on my system, and when the Syncthingtray icon reverts to the green "Idle" indicator, I know my transfer has finished.

As far as I can tell, the Syncthing web UI doesn't have any indicator for the overall status of the instance, only per-folder and per-remote-device individually, so there is no example there to follow in terms of the tray icon itself. I will note that when I drop some large-ish files into my sync folder locally, the status indicator for my remote server device does switch to an animated blue "Synchronizing" state until they've all finished transferring, so a quick glance at the top-level nothing-expanded view in the WebUI for my local Syncthing instance provides an immediate visual indication that something is actively happening and all is not in sync and idle.

It seems to me that the intuitive and sensible behaviour for the Syncthingtray icon is to only show static green "Idle" if all of the folders and devices are idle, and if any of them are busy, synchronizing, scanning, out of sync, whatever, then the tray icon should inherit that so it's always indicating the "most active" state of all the entities in its drop-down list. (Which, for the record, is how the tray icons for Dropbox, Spideroak, and Syncthing-GTK all behave. They only show the "Idle" indicator when there's nothing happening. Uploading, downloading, and scanning for changes all get some sort of "Busy" indication, and when you get a "Busy" indication that never goes away, that tells you that something is stuck in an inconsistent state and needs to be fixed.)

As far as I can tell, the Syncthing web UI doesn't have any indicator for the overall status of the instance

I made Syncthing Tray behave like Syncthing's favicon.

I would think that both "pulling changes from remote" and "pushing changes to remote" would count as "Synchronizing", because in both of those cases my local Syncthing instance is actively transferring data in order to get local and remote in sync... which is the opposite end of the scale from "Idle".

But if it is not actually pushing changes to the remote (e.g. because the remote has errors or the folder is simply paused on the remote) than "Synchronizing" is also kind of wrong while "Idle" at least represents the status of the current Syncthing instance.

Uploading, […] all get some sort of "Busy" indication […]

When I was asking the Syncthing developers a related question in the forums, I was told that "There is no concept of uploading in syncthing.": https://forum.syncthing.net/t/proper-way-for-sync-complete-notification-and-actual-idle-detection/11195/2

It seems to me that the intuitive and sensible behaviour

To sum it up: I am mainly constrained by Syncthing's limits and try to be consistent with the official Syncthing UI.

But yes, there's this simple case when you changed some file locally and want to know that one specific (or at least one arbitrary other device) is fully in sync. In this case you can obviously assume that the folder is not paused on the other device and you would likely want to take care of any errors on the remote side immediately.

So similar to the remote notifications (which are also about this use case) it would for sure make sense to have this as a configurable feature. Maybe it would be best if the status would then be "Synchronizing (downloading and uploading)" or just "Synchronizing (downloading)"/"Synchronizing (uploading)". Although "uploading" might not be the best term considering the previous explanations. Maybe "remote syncing" would be better. It would at least match the wording in Syncthing's UI.

There are some workarounds:

It is actually possible to connect to the remote directly. At least the Qt Widgets based tray can connect to multiple instances simultaneously (showing multiple icons). So one could show a 2nd tray icon via syncthingtray --connection $remote_config_name. But of course that's not very convenient.
One can enable the notification for remote directory completion. However, these notifications are not working well and I don't know how to improve them considering what I get from the API. (Well, likely there's still room for improvement.)
If you simply want to wait for the remote device's completion, you can use syncthingctl, e.g. syncthingctl wait-for-idle --url https://remote-device/syncthing --credentials […] --timeout 1800000 --at-least 5000 && systemctl poweroff

I made Syncthing Tray behave like Syncthing's favicon.

Ah, my bad; I didn't notice that the web interface had an animated favicon to show status.

But if it is not actually pushing changes to the remote (e.g. because the remote has errors or the folder is simply paused on the remote) than "Synchronizing" is also kind of wrong while "Idle" at least represents the status of the current Syncthing instance.

Hmm. Maybe I don't understand the underlying paradigm well enough here... If it is possible for my local Syncthing instance to know that the folder is paused on the remote, I would expect that to show up as some sort of little badge on the tray icon to indicate in essence "we're not actively transferring data, but this instance is not actually in sync with all of the currently connected remotes". I feel fairly strongly that it violates the principle of least surprise to show a green, idle, "all good" indicator when we're not actually in sync with all connected remote instances, regardless of whether that out of sync condition is due to changes currently being transferred from/to the local instance to/from one or more remotes, an error condition (whether the error is local or remote), or one or more remote instances having the folder paused. The fact that every other synchronizing utility I've encountered follows the paradigm of showing the "all good" idle indicator only when everything is actually in sync between local and remote reinforces my belief that that's the intuitive and expected behaviour from the point of view of most users. Having the default behaviour mask an error condition in my Syncthing setup (whether the error is local or remote) behind a steady green idle indicator that gives the impression "all is well" is dangerously misleading.

If there is no way to distinguish "remote is genuinely synchronizing by downloading from this instance" vs "remote is out of sync because of an error on their end" vs "remote is out of sync because they have this folder paused", then it seems to me that the most intuitive option would be to represent them either with a twirling "busy" state or a yellow exclamation point badge or something to indicate that all is not, in fact, in sync. Maybe it would be possible to make that notification dismissable, so in a case like yours where there's a long-standing error on the remote you could acknowledge the warning and then it would ignore that problem on that specific remote instance and show green if everything except that is in sync...

To sum it up: I am mainly constrained by Syncthing's limits and try to be consistent with the official Syncthing UI.

As for Syncthing's limits, obviously they impose hard constraints on Syncthing Tray's capabilities. On the other hand, I would respectfully submit that if the official Syncthing UI behaves in a way that is bizarre, counterintuitive, and violates a UI convention that is consistent across all other sync utilities... a departure from how the official UI does things might be the right way to go.

So similar to the remote notifications (which are also about this use case) it would for sure make sense to have this as a configurable feature.

Having a configurable feature to show something other than "all green" when any connected remote is known not to be in sync would be satisfactory from my point of view. As for how granular that is (determined by Syncthing's limitations), the specific wording of the status ("uploading", "remote syncing" etc), I have no particular preference, and honestly don't know enough about the underlying mechanisms to offer a meaningful opinion. Whether it's visually represented by twirling busy indicator, yellow exclamation badge, or (subject to what you can and can't distinguish from Syncthing itself) some combination of both, I happily defer to your expertise. I would suggest that if you do implement this feature it should be on by default, so users have to knowingly opt in to having errors and inconsistencies masked behind a green idle indicator.

Thanks for pointing out those workarounds! For now, I'll enable the notification for remote directory completion and see how it goes.

I see that there's room for improvement but at the same time I'm not sure how to improve it exactly. I don't want to change the status icon from being misleading just to being misleading in a different way. So far I usually took the official UI as reference. It might make sense to "departure from it" (as you phrased it) but that leaves me without the usual reference and in an area where the REST-API might not provide the necassary information (e.g. to find out whether a directory is paused on the remote or actually in an error state).

It would make sense to talk with the official developers about the best approach. However, I haven't pinged them so far because I personally don't find this issue very important. My main concern is usually the local device and not the rest of the possibly big cluster. In that sense I'd also like to note that if the ideal solution requires a complicated/sophisticated implementation on my side I might not be willing to take the effort.

I would personally prefer if Syncthing itself would provide an overall status which I would simply just display. That sounds better than adding this kind of logic to every GUI implementation. The Syncthing developers themselves also know much more about the internals so their solution would likely be better anyways. Maybe the Syncthing forum is a good place to have a discusson about this. (Maybe there's even already a forum post or GitHub issue for this feature.)

Having a configurable feature to show something other than "all green" when any connected remote is known not to be in sync would be satisfactory from my point of view.

Since it would never go green when a remote has paused directories that doesn't seem very useful to me. However, it is definitely a conceivable workaround in certain cases. Maybe I'll implement it at some point but due to the limited usefulness I'm not very motivated.

Okay, thanks for your thoughtful and detailed answers.

For the record, I remain convinced that — if you have to choose between the two — the misleading situation of doubling up "busy" and "something is wrong" under the same indicator is objectively and significantly less bad than the misleading situation of concealing busy/uploading and error conditions behind an all-green "idle" indicator. The former is inconveniently misleading, but the latter is dangerously misleading and invites situations such as:

"Oh, my Syncthing-tray icon is showing idle. The only sane way to read that (as reinforced by the behaviour of every other sync client out there) is that the 6 gigs of work files I dropped in the local sync folder have finished transferring to remote and I can shut down the machine now, secure in the knowledge that those files will be waiting for me when I get to my home PC."

Yes, both are misleading and can be considered failure modes, but "fail safe" is a major design consideration, not a trivial distinction.

And If "at least one first-hop remote maybe has the folder paused and maybe has an error" can be distinguished from "at least one first-hop remote is actively transferring files to/from our local instance", then wouldn't it make sense (and be fairly easy) to do something like the following?

"actively transferring" is represented by a twirling busy indicator — regardless of transfer direction; and
"maybe paused / maybe error" is represented by something like a little static yellow exclamation badge that can be overlaid on the icon regardless of whether the underlying icon is in "idle" or "busy" status.

That way, you would still get meaningful "busy" vs "not busy" indication in your situation where you have an error on remote that you haven't got around to fixing, AND the UI wouldn't be inviting data loss by concealing the states "uploading" and "we're out of sync and I don't know why" behind an all-green "in sync and idle" indicator.

However, if you're not convinced at this point, I can only suggest that we agree to disagree. Obviously it's your project, and it's your call what features are of enough interest to devote your time to them. As you suggest, maybe I will go see if I can convince the Syncthing devs to include more helpful status info in the API, so as to make the job easy on your end.

The current release allows to take the progress of remote directories into account when computing the overall status. You can now choose for each connection configuration what information should be considered to compute the overall status. In your case you would check "Remote dir has outstanding progress".

I made this configurable so the default behavior some people might prefer does not change and also other preferences like #76 can be supported.

Nice! I look forward to trying that out.

On that note, should the Appimage of v1.1.1 be in the Appimage repository for releases linked from README.md? It's still showing v1.1.0, with a modified date of Dec 2.

That's because my build scripts (contained by c++utilities) contained a bug which prevents the AppImage to build. The AppImage built from Git master (also mentioned in the README) should be up-to-date.

This should be done. Please reopen if something is missing.

Martchus / syncthingtray

Enhancement: Tray Status for Syncing Remote Machine #74