maxmind / mmdbwriter

Go library for writing MaxMind DB (mmdb) files
Apache License 2.0
116 stars 28 forks source link

Use less memory #21

Closed oschwald closed 3 years ago

oschwald commented 3 years ago

The two most notable changes in this PR are switching to sha256 for the data keys and the addition of ref counting on data in the data map so that we can delete it once it is no longer needed.

The key generation could probably use some profiling-based optimization. Right now, we are serializing the data using the same serializer used in the data section and then taking the sha256 of that. There is likely a faster way to generate a unique key.