maxmind / geoipupdate

GeoIP update client code
Apache License 2.0
726 stars 142 forks source link

Systemd timer unit for scheduled run of geoipupdate #122

Open gdsotirov opened 3 years ago

gdsotirov commented 3 years ago

I'd like to suggest adding a systemd timer unit for scheduled running of geoipupdate command. The service unit could be as simple as:

[Unit]
Description=GeoIP databases update service
Wants=geoipupdate.timer

[Service]
Type=oneshot
ExecStart=geoipupdate

[Install]
WantedBy=multi-user.target

And the timer unit itself could be:

[Unit]
Description=GeoIP databases update timer
Requires=geoipupdate.service

[Timer]
Unit=geoipupdate.service
# Monthly at 00:00 on the first day of month
OnCalendar=*-*-01 00:00:00

[Install]
WantedBy=timers.target

I found it strange that the distribution packages doesn't provide such timer unit, but I guess if it is added upstream it would benefit all.

paravoid commented 1 year ago

I found it strange that the distribution packages doesn't provide such timer unit, but I guess if it is added upstream it would benefit all.

For what it's worth, in Debian we have been shipping this for a couple of years now:

[Unit]
Description=Weekly GeoIP update
Documentation=man:geoipupdate(1)

[Timer]
OnCalendar=Wed UTC
AccuracySec=12h
Persistent=true

[Install]
WantedBy=timers.target

I believe I had it configured to check at Wed UTC, because MaxMind used to update (most of) its databases on Tuesday. I see now this has changed to Tuesday & Friday, so perhaps we're due for an update?

If MaxMind would like to ship a timer file with what they consider sane defaults, I could always switch to it of course!

paravoid commented 11 months ago

https://support.maxmind.com/hc/en-us/articles/4408216129947-Download-and-Update-Databases currently says "Every Tuesday and Friday" for all databases except Anonymous IP, and "In the event we have an unexpected delay for any of our database updates such that an update will not be available by 11:59pm US Eastern on a release day".

Based on that, I'm going to configure the Debian package to default to:

[Timer]
OnCalendar=Wed,Sat America/New_York
Persistent=true

(This is the equivalent to Wed,Sat *-*-* 00:00:00 America/New_York per systemd-analyze calendar)

Hopefully that strikes the right balance between using fresh data and not polling the MaxMind servers excessively. MaxMind folks, let me know if you'd like different defaults for the Debian package!

oschwald commented 11 months ago

Would it be possible to add a random delay to that? The primary issue we have with default timers for geoipupdate is tens of thousands of hosts making requests at exactly the same time. This is less of an issue since we have moved the downloads to Cloudflare workers and R2 for the service, but even there we do occasionally hit their very high limits, which can result in failed requests for users. I am not an Systemd expert, but perhaps something like RandomizedDelaySec=3h would work.

paravoid commented 11 months ago

Yeah that sounds very sensible. I'll add that to the next upload.

geuis commented 10 months ago

Wondering if this has been implemented yet or if its still in a dev branch?

paravoid commented 9 months ago

Yeah that sounds very sensible. I'll add that to the next upload.

This is implemented as of 6.1.0-1, uploaded today.

Note, for the casual reader, that this a Debian revision, i.e. what I (the Debian maintainer) have uploaded to Debian unstable. That statement does not mean that this change has been merged into this (upstream) repository, or, if it ever will. (In other words, this is a bit of offtopic for this issue tracker!)

oschwald commented 9 months ago

Thank you!