This is a tiny Python library for resolving a lot of DNS records asynchronously. On a speedy internet connection it managed to do 20,000 resolutions per minute.

It was written by Peteris Krumins (peter@catonmat.net). His blog is at http://www.catonmat.net -- good coders code, great reuse.

The code is licensed under the MIT license.

The code was written as a part of the article "Resolving DNS Asynchronously" on my website. The whole article can be read at:

http://www.catonmat.net/blog/asynchronous-dns-resolution/

Table of contents: [1] The problem. [2] The solution and how to use async_dns.py [3] What is slow_dns.py included in this source tree?

[1]-The-problem---------------------------------------------------------------

Once upon a time, I had to quickly resolve thousands of DNS names. My first solution was to call gethostbyname repeatedly for each of the hosts. This turned out to be extremely slow. I could only do 200 hosts in a minute. I talked with someone and he suggested to try to do it asynchronously. I looked around and found adns - asynchronous dns library written in C (see link (1) below). Since I was writing the code in Python, I looked around some more and found Python bindings for adns (see link (2) below). I tried adns and - wow - I could do 20,000 hosts in a minute!

...

Continue reading on http://www.catonmat.net/blog/asynchronous-dns-resolution/

Link references:

(1) http://www.chiark.greenend.org.uk/~ian/adns/
(2) http://code.google.com/p/adns-python/

[2]-The-solution--------------------------------------------------------------

I wrote async_dns.py Python module to do the resolution. It uses Python wrapper for adns C library (see links (1) and (2) above).

The async_dns.py module exports AsyncResolver class. The constructor of this class takes a list of domains to resolve.

Here is an example that resolves three domains asynchronously:

from async_dns import AsyncResolver

ar = AsyncResolver(
       ["www.google.com", "www.reddit.com", "www.nonexistz.net"]
)
resolved = ar.resolve()

for host, ip in resolved.items():
  if ip is None:
    print "%s could not be resolved." % host
  else:
    print "%s resolved to %s" % (host, ip)

The constructor of Asynchronously also takes 'intensity' paramter, which specifies how many hosts to resolve asynchronously at once.

Calling the resolve() member function on an AsyncResolver object starts resolving all the domains that were passed to AsyncResolver constructor.

Once it's done, it returns a dictionary of { 'host': 'ip' } structure.

[3]-What-is-slow_dns.py-in-the-source-tree?-----------------------------------

The file slow_dns.py is my first attempt at resolving a lot of DNS hosts. It uses the standard system resolver "gethostbyname" from Python's socket module.

It can only do 200 resolutions per minute.

Python module slow_dns.py exports resolve_slow function, it takes a list of hosts as an argument, resolves them, and returns a dictionary of { 'host': 'ip' } pairs.

You can use it the following way:

from slow_dns import resolve_slow

resolved = resolve_slow(['www.reddit.com', 'www.catonmat.net'])
for host, ip in resolved.items():
  if ip is None:
    print "%s could not be resolved." % host
  else:
    print "%s resolved to %s" % (host, ip)

See my original article for more information on both async_dns.py and slow_dns.py.

It's here:

http://www.catonmat.net/blog/asynchronous-dns-resolution/

That's it. Happy asynchronous resolution!

Sincerely, Peteris Krumins http://www.catonmat.net