search

ipapi-is / ipapi

Documentation for https://ipapi.is/ and repository for the Geolocation, ASN and Hosting Ranges databases.
https://ipapi.is/
89 stars 1 forks source link

Want to use Geolocation DB csv file for nginx #8

Open jkuj6 opened 7 months ago

jkuj6 commented 7 months ago

I want to use your https://ipapi.is/developers.html#geolocation-database csv file for nginx, however it seems im unable to easily convert it to .dat.

Do you have an easier way to do this?

At the moment im facing a few problems.

  1. I tried using https://github.com/ipinfo/mmdbctl to convert it however there is an issue with the CIDR notation, so i opened an issue here https://github.com/ipinfo/mmdbctl/issues/32 but no reply yet.

  2. Then i tried creating my own script to first fit the same format of the csv file as that of ipinfo, i was able to adjust the csv file, except for one issue that i noticed in your csv DB when trying to then use mmdbctl...

    mmdbctl import --in geolocationDatabaseIPv4_converted.csv --out data.mmdb
warn: couldn't insert line '154.17.129.024,154.17.129.024,154.17.0.0,Warsaw,Mazovia,Poland,52.22977,21.01178,05-077,Europe/Warsaw'
writing to data.mmdb (1278428 entries)

    This 154.17.129.024 is incorrect from your DB. id appreciate if you could fix it.

After i successfully convert it to .mmdb, i can then finally use yet another tool to convert the .mmdb into legacy .dat for usage with nginx.

Finally, to avoid so many conversion tools, Id really appreciate it if you could have your csv file ready for use with nginx, in either .dat or in .mmdb format, or even in a compatible CSV format that is easily converted with mmdbctl.

thanks in advance

t-e-s-tweb commented 7 months ago

I want to use your https://ipapi.is/developers.html#geolocation-database csv file for nginx, however it seems im unable to easily convert it to .dat.

Do you have an easier way to do this?

At the moment im facing a few problems.

  1. I tried using https://github.com/ipinfo/mmdbctl to convert it however there is an issue with the CIDR notation, so i opened an issue here is ipapi csv's conversion supported? ipinfo/mmdbctl#32 but no reply yet.
  2. Then i tried creating my own script to first fit the same format of the csv file as that of ipinfo, i was able to adjust the csv file, except for one issue that i noticed in your csv DB when trying to then use mmdbctl...
mmdbctl import --in geolocationDatabaseIPv4_converted.csv --out data.mmdb
warn: couldn't insert line '154.17.129.024,154.17.129.024,154.17.0.0,Warsaw,Mazovia,Poland,52.22977,21.01178,05-077,Europe/Warsaw'
writing to data.mmdb (1278428 entries)

This 154.17.129.024 is incorrect from your DB. id appreciate if you could fix it.

After i successfully convert it to .mmdb, i can then finally use yet another tool to convert the .mmdb into legacy .dat for usage with nginx.

Finally, to avoid so many conversion tools, Id really appreciate it if you could have your csv file ready for use with nginx, in either .dat or in .mmdb format, or even in a compatible CSV format that is easily converted with mmdbctl.

thanks in advance

were you able to make it work? I would appreciate it if you could provide the method and your scripts which you used to achieve this.

ipapi-is commented 7 months ago

I fixed the underlying root issue. Please try again.

I think I will also provide a .mmdb file, you have me convinced :)

ipapi-is commented 7 months ago

Remove first column from the database:

cut -d, -f2- geolocationDatabaseIPv4.csv > data.csv

Then convert:

mmdbctl import --in data.csv --out data.mmdb

that works like a charm and yields:

writing to data.mmdb (1347178 entries)