DB-IP Subscription account support

[PDD777]

Hi,

Thank you for writing this, it has helped me a lot.

I do have an idea, not sure if you've previously had this asked or explored, but supporting the use of a subscribed csv download via db-ip would be nice.

Something along the lines of what db-ip have documented for their php updater(https://db-ip.com/tutorials/database-file-update).

So having a 'config' file which holds the url, filename etc that the script then use in producing the nft mapping files.

I'm happy to help in testing this etc, as we have an active subscription now, given that the free version of the data only supports down to /24 granularity on IPv4 addresses.

Thank you Paul

[pvxe]

Hi Paul, I'm glad this script was helpful to you. I didn't know about a php script from db-ip used to update and download its databases.

The part about having a config file makes total sense, it is something that has been missing since the beginning. Even if just to hold its current parameters.

In the other hand supporting the php script inside nftables-geoip would add some "assumptions" which I think might fall outside the scripts intention, such as the user having a correct php installation. In addition to that, I'm not a fan of adding new things by means of just calling a subprocess because the nftables-geoip would have a more difficult time to deal with an error if it only can know the exit code. But nonetheless I understand the appeal, it could be added as a separate component tied to an specific parameter, e.g: --use-updater / -u.

If your main goal is to be able to download other databases apart from the free tier, which are other mechanisms to do so? Maybe a different approach could be added to nftables-geoip so you can fetch better databases.

In any case I feel like I could use a bit more details regarding your idea, please feel free to expand on that. A use case example usually works best.

Cheers!

[PDD777]

Hey Jose,

Cool, maybe just keeping it a cli py to do downloading and mapping, but have either a config file(in which I think would make more sense) or parameters like --url or something.

The py process is working well, wouldn't want to change that, just maybe expand it do dl "non-free" csvs.

The subscription accounts also have mmdb(maxmind) db formats too, and can do daily or monthly downloads, depending how much you pay.

So for an example, I've created a cron job that fires the py script each night. nft_geoip.py -d -c cn,ru,in,sy,iq,ir,id -o /path/to/firewalls/ips/

In the nft_geoip.py I've modified the url and NetworkEntry array;

entries in DB-IP geoip csv <- hehe markdown headers....

NetworkEntry = namedtuple('NetworkEntry', 'network_first, ' 'network_last, ' 'continent_code, ' <- extra field from paid version 'country_alpha_2')

url = 'https://download.db-ip.com/key/${paidsubscriptionkeygoeshere}.csv'.format(time.strftime("%Y-%m"))

These 2 changes work perfectly, script runs without fault.

Or server is running Deb12 too, which has change how it handles python and pip installs etc too.

Hope this helps Paul.

[PDD777]

Hi, Just an update to the above post, the url can be pulled as follows, for paid subscriptions.

url = requests.get('https://db-ip.com/account/yourpaidaccountidnumberkey/db/ip-to-country/').json()["csv"]["url"].format(time.strftime("%Y-%m"))

This will give you the 'https://download.db-ip.com/key/rotatingdailykeygoeshere.csv' url to grab the csv file.

The script then proceeds nicely to produce the nft maps.