what would be required to support a few more databases?

mteodoro / mmutils

Tools for working with MaxMind GeoIP csv and dat files

MIT License

95 stars 47 forks source link

what would be required to support a few more databases? #3

Closed convertstar closed 10 years ago

convertstar commented 10 years ago

I'm really looking to use this utility to modify info from GeoIP Country (GEO-108) and GeoIP ISP (GEO-124) csv files and produce the corresponding dat files. For the country we only access the .country_code_by_addr and for the isp we only access the .org_by_addr methods of the python api.

Please, any info or help would be appreciated. Thank you for making this utility available in the first place.

mteodoro commented 10 years ago

I just committed support for GeoIP Country and Country v6 databases in f9d2338d091cd115e8417b15130ad2325d9bc3fa. Unfortunately I don't have a copy of an ISP .dat/csv pair which makes it hard to add support for that.

convertstar commented 10 years ago

Hi Mark,

If we sent you $25 via paypal, could you buy the ISP database? I don't think we could just send it because of our license agreement with them.

Thanks Gavin

On Wed, Feb 5, 2014 at 5:42 PM, Mark Teodoro notifications@github.comwrote:

I just committed support for GeoIP Country and Country v6 databases in f9d2338https://github.com/mteodoro/mmutils/commit/f9d2338d091cd115e8417b15130ad2325d9bc3fa. Unfortunately I don't have a copy of an ISP .dat/csv pair which makes it hard to add support for that.

Reply to this email directly or view it on GitHubhttps://github.com/mteodoro/mmutils/issues/3#issuecomment-34283946 .

Gavin Aitken CEO and Co-Founder Go2mobi / ConvertStar Inc. p. 888-353-2442 ext.700 c. 250-661-5256 e. gavin@convertstar.com

mteodoro commented 10 years ago

Actually the ISP and Org db formats look nearly identical to the ASN format that's already supported. (The only difference is 4-byte records instead of the usual 3-byte records). I just committed support for them in ecf77fb6771ac875475dac0148c14590a32213f7 in the isp-org branch.

Would you mind trying it on your ISP database and letting me know if it works? To build the database, run:

$ csv2dat.py -w mmisp.dat mmisp GeoIPISP.csv

If you have a file of IPs you can sanity check the generated .dat against the original .dat with:

$ csv2dat.py test GeoIPISP.dat mmisp.dat ips.txt

convertstar commented 10 years ago

Awesome thanks.. We'll give it a try and let you know. Thanks Gavin On Feb 5, 2014 8:06 PM, "Mark Teodoro" notifications@github.com wrote:

Actually the ISP and Org db formats look nearly identical to the ASN format that's already supported. (The only difference is 4-byte records instead of the usual 3-byte records). I just committed support for them in ecf77fbhttps://github.com/mteodoro/mmutils/commit/ecf77fb6771ac875475dac0148c14590a32213f7in the isp-org branch.

Would you mind trying it on your ISP database and letting me know if it works? To build the database, run:

$ csv2dat.py -w mmisp.dat mmisp GeoIPISP.csv

If you have a file of IPs you can sanity check the generated .dat against the original .dat with:

$ csv2dat.py test GeoIPISP.dat mmisp.dat ips.txt

Reply to this email directly or view it on GitHubhttps://github.com/mteodoro/mmutils/issues/3#issuecomment-34290523 .

convertstar commented 10 years ago

Hi Mark,

I tested the ISP handling code in commit ecf77fb6771ac875475dac0148c14590a32213f7 and all seems well. I produced the .dat file from the .csv file with no issues and then ran the test code and it had no issues either, checked 20k IPs and all matched test vs reference. Thanks a ton for adding this support to the utility.

Chris

mteodoro commented 10 years ago

Sounds good. I merged this to master in #4.