search

msimerson / maxmind-geolite-mirror

Mirror MaxMind GeoIP dbs from geolite.maxmind.com
https://www.npmjs.com/package/maxmind-geolite-mirror
MIT License
10 stars 5 forks source link

test downloaded files and assure correct format #18

Closed msimerson closed 6 years ago

msimerson commented 6 years ago

MaxMind sometimes provides files in .tar.gz format. This library has gained expansion support for it in PR #16 but we should have a test that the output result is a proper MaxMind mmdb file.

One way to accomplish this is to include one of the MaxMind libraries as a dev dependency and then have it load the downloaded file(s). That's a great end-to-end test, but also pretty heavy. I'd rather tests that heavy (and that hit MaxMind servers) be reserved for running only on developers workstations.

A good unit test would be including a couple truncated (only the file header would matter) data files in test/fixtures and using an equivalent of the file utility to taste them.

good

$ file GeoLite2-City.mmdb 
GeoLite2-City.mmdb: data

bad

$ file GeoLite2-ASN.mmdb 
GeoLite2-ASN.mmdb: POSIX tar archive (GNU)
msimerson commented 6 years ago

Looks like the file-type module would be a good candidate for tasting the file signature.

sbma44 commented 6 years ago

An even simpler approach could be to just stick save some known strings into fake mmdb files and either gzip or tar them up into fixtures. No need for a dependency, I don't think. If this sounds ok I can take a pass at it.

msimerson commented 6 years ago

I think you're saying about the same as I am: using generated "fake" mmdbs with the valid file signature for a tar file (this is hex, not text) seems like it ought to be robust.

If you use the utility strings on the .mmdb that's actually a tar file, I saw text strings in it. That doesn't seem quite good robust enough.

msimerson commented 6 years ago

fixed by #20