Trying to understand the output

Hi,

Thanks for this cool program :) I was trying to do a "dig" for each domain manually, and it was bloomin slow. So I'm using:

./massdns/bin/massdns -r massdns/lists/resolvers.txt -o S -t A massdns/domains-to-do.txt > results.txt

Which finishes with:

Processed queries: 239005
Received packets: 405301
Progress: 100.00% (00 h 00 min 57 sec / 00 h 00 min 57 sec)
Current incoming rate: 24 pps, average: 7054 pps
Current success rate: 1 pps, average: 4160 pps
Finished total: 239004, success: 239002 (100.00%)
Mismatched domains: 17138 (4.29%), IDs: 0 (0.00%)
Failures: 0: 19.28%, 1: 16.90%, 2: 13.38%, 3: 10.62%, 4: 8.31%, 5: 6.52%, 6: 5.10%, 7: 3.94%, 8: 3.12%, 9: 2.47%, 10: 2.01%, 11: 1.59%, 12: 1.27%, 13: 1.02%, 14: 0.82%, 15: 0.67%, 16: 0.57%, 17: 0.45%, 18: 0.35%, 19: 0.31%, 20: 0.24%, 21: 0.20%, 22: 0.15%, 23: 0.13%, 24: 0.10%, 25: 0.08%, 26: 0.06%, 27: 0.05%, 28: 0.05%, 29: 0.04%, 30: 0.04%, 31: 0.02%, 32: 0.02%, 33: 0.02%, 34: 0.01%, 35: 0.02%, 36: 0.01%, 37: 0.01%, 38: 0.01%, 39: 0.01%, 40: 0.01%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 0.00%,
Response: | Success:               | Total:
OK:       |       229527 ( 96.04%) |       242872 ( 60.73%)
NXDOMAIN: |         1643 (  0.69%) |         1676 (  0.42%)
SERVFAIL: |         7832 (  3.28%) |         9740 (  2.44%)
REFUSED:  |            0 (  0.00%) |       145641 ( 36.42%)
FORMERR:  |            0 (  0.00%) |            0 (  0.00%)

My file has 239,005 domains in - but I see quite a lot of SERVEFAIL and NXDOMAIN errors (over 9k of them). What do those mean? I'm going to run a test to compare what is outputted, vs what I'm passing in - but the outputted results.txt file is only 229,641 lines, which makes me think quite a few are missing - especially when a lot of domains have multiple lines:

aradhana.org.uk. A 185.230.63.107
aradhana.org.uk. A 185.230.63.186
aradhana.org.uk. A 185.230.63.171

Thanks in advance!

Andy

OK so here is an example:

nordoff-robbins.org.uk

Comes back ok with a normal "dig":

dig nordoff-robbins.org.uk +short | paste -d " " - - - 84.207.248.85

Yet with massdns, it doesn't include it in the results. However, when I try again with just that one domain, it works:

Response: | Success:               | Total:
OK:       |            1 (100.00%) |            1 (100.00%)
NXDOMAIN: |            0 (  0.00%) |            0 (  0.00%)
SERVFAIL: |            0 (  0.00%) |            0 (  0.00%)
REFUSED:  |            0 (  0.00%) |            0 (  0.00%)
FORMERR:  |            0 (  0.00%) |            0 (  0.00%)

nordoff-robbins.org.uk. A 84.207.248.85

Am I doing something wrong? Is there a "retry" rate, where it will try x number of times?

I would recommend you confirm that the resolvers you're using are in-fact public recursive resolvers. All of them. When troubleshooting, use --predictable to cause the same resolvers to be used for the same host for each session

If you're hammering resolvers too aggressively (especially those that are not intended to be used by the general public) you can see all sorts of unexpected behavior- NXDOMAIN, SERVFAIL, REFUSED. It depends on how the resolver is configured
It's also possible that you're causing the authoritative nameserver for nordoff-robbins.org.uk to receive too much load, as the requests will ultimately end up there unless your resolvers have them cached

The latter case should work itself out after a few sessions, especially if your resolvers.lst is small and/or you're using --predictable as the question will have the answers cached the second time around

I can tell you with certainty the problem is not with massdns. My suggestions for you:

Try using with a single CONFIRMED PUBLIC resolver in your resolvers.lst first
Use --predictable whenever you're trying to identify issues

Also, as a general tip, when using dig to reproduce something, you should explicitly specify a recursive resolver on the commandline. You should also explicitly specify the question type. The default for massdns is A. I know dig will return A records, but it will do other things depending on the response (or lackthereof)

For an accurate reproduction of the above, you should use this, it will test each one of your resolvers

for r in $(cat resolvers.lst); do
    dig -t A +short nordoff-robbins.org.uk +short @$r
done

Using this short loop alone may reveal that you have a bad resolver in your list

Of course, it will not reproduce the load that you're producing when using massdns

blechschmidt / massdns

Trying to understand the output #114