maxmind / MaxMind-DB-Writer-perl

Create MaxMind DB database files
https://metacpan.org/release/MaxMind-DB-Writer/
Other
77 stars 27 forks source link

Search tree duplicated via Writer has much bigger size than an original #117

Closed p01arbear closed 3 years ago

p01arbear commented 3 years ago

Hello.

I have a test script, which simply duplicates database via iterate_search_tree() => Network->new_from_integer() => insert_network() => write_tree().

Script is runned inside docker, Dockerfile.

This is a test GeoLite2-City.mmdb database.

I can't understand why the file written by the MaxMind::DB::Writer::Tree has a 3.5x much bigger search tree size than the original.

Is something wrong in the script itself or the Perl Writer API has a bug?

oschwald commented 3 years ago

You are likely creating multiple copies of the IPv4 search tree. alias_ipv6_to_ipv4 will use aliases instead, although you will likely end up with errors about inserting into aliased nodes. I don't think the Perl reader provides a way to skip aliased nodes. Personally, I would suggest looking at the Go reader and writer instead; it will also be much, much faster.