steilerDev / icloud-photos-sync

One-way sync engine for the iCloud Photos Library into the native file system
https://icps.steiler.dev/
GNU General Public License v3.0
349 stars 9 forks source link

Native support for health check ping urls #577

Open timvahlbrock opened 1 week ago

timvahlbrock commented 1 week ago

Describe the ultimate goal you want to achieve

Compliments/Background Thanks for this great library, I'm finally able to create automatic backups of the 20 year old photo library of my family, will still having all the nice perks of iCloud Photos. Have it running on my K8s Homeserver, setup was pretty straightforward. Even managed to set up a process to simplify the token renewal each month.

This great library supports setup as a daemon process or can alternatively be setup with other mechanisms to make it fully autonomous (apart from the monthly token refresh). It would be great to have the possibility to easily monitor the status of the automation process.

How do you think the feature should be implemented

The backup tool I use, borgmatic allows to set health check ping URLs. When the backup is done the configured URL is queried, if the health check hoster does not receive a ping, or the request is appended with a non-zero number (the exit code), the health check provider can notify the user of the missing or failed backup, for example via email. The health check ping can be enhanced with more detail, by adding the logs in the request body, which can then be included in the notification or email. This is currently still possible, but with a little more effort and tweaking. For my setup I overwrote the entry point of the container with the following commands.

exec 2>&1;
set -o pipefail;
icloud-photos-sync sync | tee /tmp/icloud-photos-sync.log;
exitCode=$?;
curl -fsS -m 10 --retry 5 --data-binary @/tmp/icloud-photos-sync.log https://hc-ping.com/the-healthcheck-slug/$exitCode;
exit $exitCode

While doing the health check ping manually provides a little more flexibility (for example I actually added a few more lines to include the number of downloaded photos and the consumer storage), the configuration could be a little simpler if the url can just be included in the configuration.

I would be willing to provide a pull request for this.

Checklist

steilerDev commented 1 week ago

Hey @timvahlbrock - thanks for the nice words.

Have you seen the sync metrics capability which can be enabled through a flag.

Anything missing from this that you‘d need to achieve your goals?

timvahlbrock commented 1 week ago

Well having the data easily accessible is nice, but as far as I can tell I still need to monitor myself, whether an automated run succeed or not, right?

steilerDev commented 1 week ago

Yes - the data is written to a file that you'd need to watch (I'm using telegraf to collect the data and Grafana to display and alert)

Opposite to that you are looking for a webhook functionality, right?

I've never designed such a feature, so implementing this will require an understanding of how much customisation it needs to allow in order to be useful. Nevertheless, feel free to propose a design, in case you have experience or a good idea :)

timvahlbrock commented 1 week ago

Well, I would do it pretty much like borgmatic (see link above): a single additional parameter that is the URL to ping.

timvahlbrock commented 1 week ago

Drafted together a quick PoC PR, haven't tested it yet and I would also like to include the log in the send pings. But on changes to the configuration the PR should be complete.