search

msimerson / maxmind-geolite-mirror

Mirror MaxMind GeoIP dbs from geolite.maxmind.com
https://www.npmjs.com/package/maxmind-geolite-mirror
MIT License
10 stars 5 forks source link

bin/maxmind-geolite-mirror making many queries, ending up in error 429 #47

Open DoobleD opened 3 months ago

DoobleD commented 3 months ago

Running bin/maxmind-geolite-mirror once gives the following error:

MAXMIND_LICENSE_KEY=>obfusacated>  ./bin/maxmind-geolite-mirror 
downloading /usr/local/share/GeoIP/GeoLite2-Country.mmdb
HEADERS: {"date":"Sat, 22 Jun 2024 20:56:34 GMT","content-type":"text/plain;charset=UTF-8","content-length":"44","connection":"close","cache-control":"private, max-age=0","expires":"Sat, 22 Jun 2024 20:56:34 GMT","strict-transport-security":"max-age=31536000; includeSubDomains; preload","server":"cloudflare","cf-ray":"897f2551db459ec2-CDG"}
response code 429 not handled!
exiting prematurely.

This has been tested with a fresh clone of the repo, and a fresh Maxmind GeoLite2 account with a new license key. Following the error we can see on the Maxmind account that 65 queries were made, which exceeds the 30 queries per day limit for GeoLite accounts.

image

Maxmind also sends the following email:

image

msimerson commented 3 months ago

I'm aware, I just haven't dug deep enough to figure out exactly why.

msimerson commented 3 months ago

Also, a workaround if you're using this with MT6 is to set GEOIP_UPDATER=geoipupdate in mail-toaster.conf, and use the maxmind client.

keithws commented 1 month ago

I am also receiving this issue.

In my situation, the script (run at 6am by cron) is able to make 289 requests within about 20 seconds for the exact same file (the country file) over and over again.

msimerson commented 1 month ago

Yes, the retry it's because Maxmind changed how they redirect clients after this was written. Some time needs to be spent handling them better.