Closed hacdias closed 3 years ago
Also cc @Kubuxu
@hacdias how many entries there are in IPv6 dataset?
@Kubuxu about 431 990 for the IPv6 dataset and 3 252 943 for the IPv4 one.
Pinging @lidel and @Kubuxu 😅
Sorry (got a lot on my head). The best structure for it would be ipld based radix tree, but it wouldn't be easy to implement. We can do away with something simpler (pseudo radix tree). I will write something more on that tomorrow.
Thanks! Will be awaiting!
Not accounting for IPv6 addresses, this is my current idea and plan. I am thinking of keeping the same binary tree structure as we have now, but I do have some doubts and questions to be answered. Please let me know what you think:
Feedback? @lidel @Kubuxu
Sadly I had no bandwidth, this needs another set of eyes.
Figuring out new data structures will be time consuming, and given limited bandwidth I propose we simply convert the new CSV dataset to the format of old CVS source files:
That way ipfs-geoip does not need to be rewritten, and we can at least update IPv4 info.
While planning to move to the new dataset (see #63), I found some problems I need help with!
Field, information, field and more information
First of all, right now, we have the following data for each location:
The new datasets contain much more than that:
I am pretty sure we don't need all of those fields, so the first goal of this issue is to define which informations do we want to provide through this package.
IPv6
The second issue is: how to support IPv6 (#60)? The newest dataset has an IPv6 table too! Just like the IPv4, we are provided with CIDR addresses that allow us to know the range for which to check for IPv6 addresses. However, unlike IPv4, there's no "int long" form of IPv6 so we can't keep the same structure as we have now for IPv4.
Knowing this, how'd you suggest to tackle this issue? How to organize the information in such a way we can fetch it quickly?
Languages?
The new dataset provides translations for just some languages. Are they worth including or shall we keep just the english ones for now?
Also, I am thinking about setting up a way of updating the geoip database automatically since they update it every tuesday. It would be great so we wouldn't need to think a lot about this (perhaps just merging a PR with the newer CID).
Ping @lidel