search

oschwald / maxminddb-golang

MaxMind DB Reader for Go
ISC License
576 stars 99 forks source link

Add DecodeLength and Size() #75

Open arp242 opened 3 years ago

arp242 commented 3 years ago

The use case for this is pretty much as in the test: I'd like to get the data section of a database file. As far as I know there's no easy way to do this currently.

I'm not super-happy with the name "DecodeLength()", but I couldn't think of anything better.

Size() is needed because otherwise we don't know if we're at the last entry.

oschwald commented 3 years ago

What about an iterator over the data section instead? This could encapsulate the logic for checking the section size. It could follow a similar pattern to the Networks iterator. I think the only time DecodeLength would really be useful is if you are decoding the whole data section.

I do agree that the name DecodeLength is a bit misleading, as it has little to do with the length of the data decoded as most databases will contain many pointers in this data causing the actual length of the data to be much longer.

If this was changed to an iterator, perhaps this verification code could be updated to use it:

https://github.com/oschwald/maxminddb-golang/blob/2b2ff7ada5e406deab64fc0e729837caea5663da/verifier.go#L139-L159

arp242 commented 3 years ago

I think the only time DecodeLength would really be useful is if you are decoding the whole data section.

Yeah, probably; I was a bit surprised that it's not already in here, because as far as I know this is pretty much the easiest way to get a list of ISO-3166-2 names (so many lists on the 'net are very old, have encoding errors, drop diacritics, or just have outright bad data; sigh).

Anyway, I'll have a look and change it to an iterator; can also name it something more sensible then.

arp242 commented 3 years ago

Okay, I changed it to a Data() method which returns a Data iterator, behaving pretty much the same as the Networks one.

arp242 commented 3 years ago

Hey @oschwald; is there any chance you can take another look and let me know if you want any further changes? No worries if you don't have time, I can keep using my fork (also for the related PR at https://github.com/oschwald/geoip2-golang/pull/68). Thanks!