Proposal: Option for suppressing the output of specify response types (e.g. NXDOMAIN)

[mzpqnxow]

While I realize there is definitely value in logging NXDOMAIN responses, depending on the users' goals, logging NXDOMAIN may be problematic

For example, when surveying a set of a several thousand domains to simply determine the existence of an A record with a brute-force/enumeration list of 100k+ per domain, this results in a ton of disk i/o (and disk space usage) when it may not be useful

Would you consider having a command-line option that would allow the user to specify one or more specific response codes and have logic to sinkhole such responses rather than log them to stdout/stderr/disk? I glanced at the code and I think it's a minor change, essentially just a quick conditional before the fprintf()`, taking a list of which responses types to ignore from the context holding the commandline arguments

I know this can be done by piping the output through grep -v, but it would be nice to be able to specify it when logging the output to a file natively within massdns. It also technically reducing some cycles since fprintf() goes through libc, does some locking and blabla.. so it's a little bit better than filtering on the way out.

I'm happy to have a go at a PR for this if you would accept it. By default, it would filter no records. One or more types could be specified on the command-line as comma separated values, or by using the option more than once, e.g. --ignore NXDOMAIN --ignore FORMERR (probably in most cases you would want to know about FORMERR responses, but this is just an example)

I find a common pattern for my use is as follows:

1. Generate brute force FQDN lists for each domain I am authorized to enumerate
2. Run massdns and log to NDJSON format
3. Select all NOERROR responses via JQ and put the results to a separate file in a specific format for post-processing
4. Select all SERVFAIL responses via JQ and re-run with these (since often SERVFAIL responses are ephemeral and are "fixed" using multiple passes, probably due to some throttling occurring between the recursive resolver and the authoritative NS that was triggered in the first run)
5. Repeat 3 and 4 a few times until SERVFAIL totals are constant

In this specific process, NXDOMAIN records are completely ignored and just taking up space- sometimes a significant amount of space. The only use I have for NXDOMAIN in my use-case/process is when testing correctness of recursive resolvers, though I tend to do that outside of massdns anyway

Thoughts?

[blechschmidt]

Thanks for the suggestion.

One or more types could be specified [...] by using the option more than once, e.g. --ignore NXDOMAIN --ignore FORMERR (probably in most cases you would want to know about FORMERR responses, but this is just an example)

This is implemented now by e5be8fae9b8bfc9deaabaff21c076eb4fa95199e.

By the way, I have also pulled 1ed2cf267f9c1fef00ec6d8059d814bbc5515100. Thank you for spotting this.

[mzpqnxow]

Killer, thanks!