search

elceef / dnstwist

Domain name permutation engine for detecting homograph phishing attacks, typo squatting, and brand impersonation
https://dnstwist.it
Apache License 2.0
4.93k stars 775 forks source link

Make WHOIS lookup fail gracefully #231

Open WilliamDEdwards opened 1 month ago

WilliamDEdwards commented 1 month ago

When --whois is enabled, and one lookup fails, dnstwist dies with an exception such as:

Traceback (most recent call last):
  File "/usr/local/bin/dnstwist", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1182, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1540, in run
    wreply = whois.whois('.'.join(domain_tld(domain['domain'])[1:]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 258, in whois
    return self._extract(self.query(domain, server))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 253, in query
    return self.query(query, refer.group('server'))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 237, in query
    sock.connect((server, 43))
socket.gaierror: [Errno -2] Name or service not known

As you can see, the WHOIS server 'refers' to another. In this case, it's whois.iana.org, and I don't know why it fails. But that's irrelevant to this issue; I believe such a situation should be handled gracefully, by skipping the domain silently and logging this to the debug log. After all, not all registries provide reliable WHOIS servers.

WilliamDEdwards commented 1 month ago

Upon further inspection, (at least) .ph apparently returns an empty 'refer':

refer:

For comparison, for .com:

refer:        whois.verisign-grs.com

This -obviously- confuses dnstwist.

elceef commented 1 month ago

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

elceef commented 1 month ago

Meanwhile I pushed some changes which hopefully should resolve this.

WilliamDEdwards commented 1 month ago

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

Did you see my follow-up comment (https://github.com/elceef/dnstwist/issues/231#issuecomment-2380613556)?

elceef commented 1 month ago

Empty refer fields are skipped. Initially domain name for IANA server is resolved which seems to be the cause in your network. Commit 464bb425462d393a9e074a04e5c431985072e705 should resolve it.