search

blechschmidt / massdns

A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)
GNU General Public License v3.0
3.05k stars 456 forks source link

Massdns does not respect concurrent lookups parameter #136

Closed h49nakxs closed 1 year ago

h49nakxs commented 1 year ago

Hi there,

When setting a low number of concurrent lookups with a short list of resolvers, massdns doesn't cap the concurrent lookups and use a much higher throughput.

For example, the following command : massdns -t A -s 10 -r resolvers.txt -o S -w output.txt domains.txt

with a "resolvers.txt" file containing 1 resolver will execute around 500 concurrent lookups. The same command with a "resolvers.txt" file containing about 30 resolvers will execute around 150 concurrent lookups.

If I increase the --hashmap-size number, the multiplication factor remains the same. If I use the whole list of resolvers from https://public-dns.info/, the number of concurrent lookups I've set is respected.

I suspect this is related to how the hashmap works and I'm not sure it's easily fixable.

blechschmidt commented 1 year ago

How do you measure concurrency? Are you maybe referring to the queries per second (qps) rate? (MassDNS currently does not support limiting that.) If resolvers.txt contains one resolver, -s is 10 and the round trip time for one query is 20 ms, then 500 qps are expected. The hash map is initially filled with 10 names and as soon as one name has been resolved, the next name to be resolved is pulled into the hash map.

h49nakxs commented 1 year ago

Yes I was referring to the queries per second, so my understanding wasn't correct.

Thanks for the reply.

Just for my personal knowledge : why does increasing the number of resolvers lower the qps ? Is it because of a higher RTT for some resolvers ?

blechschmidt commented 1 year ago

Yes, some resolvers have a higher RTT and some will enforce rate limits.