Structured logging

[martenlehmann]

• Program: Authoritative, Recursor
• Issue type: Feature request

Short description

I would be great if PowerDNS would log in a structured way. This means, instead of the current free-form text in an ever growing log file, events

• would contain structured details
• could be logged in JSON
• could be retrieved by exact patterns with e.g. catj instead of lose string searches with lots of false positives
• could be aggregated and processed with Logstash, Loggly etc.

Expected behaviour

1. The function signature void Logger::log(const string &msg, Urgency u) would take an additional dictionary.
2. Calls like

   g_log<<Logger::Error<<"Aborting possible open transaction for domain '"<<domain<<"' AXFR"<<endl;

   would translate into a just very concise event message and instead provide all details using a specific, clear classification like

   {
   'context': 'axfr',
   'type': 'transaction',
   'status': 'abort',
   'domain': 'whatever.com'
   }

   via the dictionary.

Actual behaviour

Free-form log messages, which are human readable, but not really machine parsable.

Other information

There are many convincing statements about why structured logging is the preferred way out there, for example:

• https://www.loggly.com/blog/why-json-is-the-best-application-log-format-and-how-to-switch/
• https://stackify.com/what-is-structured-logging-and-why-developers-need-it/
• https://blog.treasuredata.com/blog/2012/04/26/log-everything-as-json/

Usecase

At a professional scale (like Anycast setup), there's no time to watch at individual log files, even logging to syslogd is not considered state-of-the-art any longer.

Converting all free-form log messages to structured logs in PowerDNS is definitely requiring less work than asking all users to find regular expressions capturing each possible log message correctly.

[omoerbeek]

Done for rec