PowerDNS / pdns

PowerDNS Authoritative, PowerDNS Recursor, dnsdist
https://www.powerdns.com/
GNU General Public License v2.0
3.72k stars 912 forks source link

Authoritative ringbuffers are ASCII & slow #7373

Open ahupowerdns opened 5 years ago

ahupowerdns commented 5 years ago

In the Authoritative Server, we store queries and responses in several ringbuffers. These ringbuffers are templatized and could contain anything we want. As it stands, they contain strings. This means that every query goes through several DNSName::toLogString() conversions.

Under high load, this represents around 5%-10% of what we are doing. Ringbuffers with DNSNames are updated from several places:

common_startup.cc:    S.ringAccount("queries", P->qdomain.toLogString()+"/"+P->qtype.getName());
packethandler.cc:  S.ringAccount("noerror-queries",p->qdomain.toLogString()+"/"+p->qtype.getName());
packethandler.cc:    S.ringAccount("servfail-queries",p->qdomain.toLogString());
packethandler.cc:    S.ringAccount("servfail-queries",p->qdomain.toLogString());
responsestats-auth.cc:      S.ringAccount("nxdomain-queries",p.qdomain.toLogString()+"/"+p.qtype.getName());
responsestats-auth.cc:    S.ringAccount("unauth-queries",p.qdomain.toLogString()+"/"+p.qtype.getName());

To fix, templataize these ringbuffers to they contain pair<DNSName,QType> & adjust API and webserver so they do the conversion if needed.

zeha commented 5 years ago

Quoting @Habbie on a very insightful comment:

<Habbie> so writing to them is slow, but that means this does not suddenly hurt operations when you start looking at them
rgacogne commented 5 years ago

Aren't we much more likely to write to them than to read from them, though?

zeha commented 5 years ago

Summary of some FD:

Result: deferring the string work to printing will likely have very little operational impact at print time, and would save a ton of work when writing.

Related discoveries: the current code always does a by-string lookup to find the same three ringbuffers. That should also go away.

rgacogne commented 5 years ago

Mostly fixed by #7503, except perhaps that last discovery:

the current code always does a by-string lookup to find the same three ringbuffers. That should also go away.