search

gdnsd / gdnsd

Authoritative DNS Server
http://gdnsd.org/
GNU General Public License v3.0
483 stars 67 forks source link

Only read a geoip db file once #231

Open danhanks opened 1 year ago

danhanks commented 1 year ago

If the gdnsd config references the same geoip db file multiple times in the config, it re-reads it from the filesystem for each mention in the config. For configs with large geoip databases and many references to the file, this can make for long start/reload times.

Ideally, gdnsd would only need to load an individual geoip database once, then re-use the same data if referenced multiple times by different maps. I suspect this would make for less memory usage by the daemon as well.

blblack commented 1 year ago

In the short term, we can't do much to optimize this due to the existing design. The actual (as in Maxmind) GeoIP database is only loaded temporarily in order to walk the data and transform it according to config into a different internal form which is optimized based on the network boundaries implied from your config (for example, merging what might be 32 separate-but-adjacent subnets from MaxMind which all give the same result in your config, into a single larger supernet in the internal database). The part where libmaxminddb temporarily loads the input database into memory is only a small part of the time cost -- most of it is spent in the transformations which are specific to your distinct maps.

In the long run, better ways of attacking this are coming. gdnsd-4.x will retain the current plugin / geoip / etc parts, but will deprecate them all in favor of a replacement system (which will probably be the only system in 5.x). The new system will be much simpler on the daemon's side of things, and be compatible with future DNSSEC support, and will also push most of the complexity out to user-controlled scripts, including the part about parsing and optimizing anything from a Maxmind database or similar. We'll probably ship a sample script that does something approximately similar to the current plugin_geoip, but this opens up a lot of new ways to customize things. The user scripts will resolve it all down to just an optimized "192.0.2.0/24 => [x, y, z]" map to feed into gdnsd. Design is still being iterated on, but should come together over the next couple of months!