--whois parameter doesn't work

[domderen]

Hey,

It looks like there's some issue with running the --whois parameter in the newest version (20230402). I get this error:

$ dnstwist --registered --whois dnstwist.it
     _           _            _     _
  __| |_ __  ___| |___      _(_)___| |_
 / _` | '_ \/ __| __\ \ /\ / / / __| __|
| (_| | | | \__ \ |_ \ V  V /| \__ \ |_
 \__,_|_| |_|___/\__| \_/\_/ |_|___/\__| {20230402}

permutations: 100.00% of 3790 | found: 2 | eta: 0m 00s | speed: 14 qps
WHOIS: dnstwist.it (50.00%)we have it
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 1295, in run
    whoisq = whois.query('.'.join([dom, tld]))
  File "/usr/local/lib/python3.10/site-packages/whois/__init__.py", line 262, in query
    whois_str = do_query(
  File "/usr/local/lib/python3.10/site-packages/whois/_1_query.py", line 70, in do_query
    _do_whois_query(
  File "/usr/local/lib/python3.10/site-packages/whois/_1_query.py", line 171, in _do_whois_query
    p = subprocess.Popen(
  File "/usr/local/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'whois'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dnstwist", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 943, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 1297, in run
    if args.debug:
AttributeError: 'Namespace' object has no attribute 'debug'

I wasn't able to reproduce it on MacOS, but I did manage to reproduce it in python:3.10-slim docker image. Below I attach minimal reproduction steps:

$ docker run -it python:3.10-slim bash

root@89b4a37fa045:/ pip freeze
<empty response, no packages installed yet>

root@89b4a37fa045:/ python --version
Python 3.10.11

root@89b4a37fa045:/ pip install dnstwist[full]
...
Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 attrs-22.2.0 certifi-2022.12.7 charset-normalizer-3.1.0 dnspython-2.3.0 dnstwist-20230402 frozenlist-1.3.3 geoip2-4.6.0 idna-3.4 maxminddb-2.2.0 multidict-6.0.4 ppdeep-20200505 requests-2.28.2 tld-0.13 urllib3-1.26.15 whois-0.9.27 yarl-1.8.2

root@89b4a37fa045:/ dnstwist --registered --whois dnstwist.it
     _           _            _     _
  __| |_ __  ___| |___      _(_)___| |_
 / _` | '_ \/ __| __\ \ /\ / / / __| __|
| (_| | | | \__ \ |_ \ V  V /| \__ \ |_
 \__,_|_| |_|___/\__| \_/\_/ |_|___/\__| {20230402}

permutations: 100.00% of 3790 | found: 2 | eta: 0m 00s | speed: 14 qps
WHOIS: dnstwist.it (50.00%)we have it
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 1295, in run
    whoisq = whois.query('.'.join([dom, tld]))
  File "/usr/local/lib/python3.10/site-packages/whois/__init__.py", line 262, in query
    whois_str = do_query(
  File "/usr/local/lib/python3.10/site-packages/whois/_1_query.py", line 70, in do_query
    _do_whois_query(
  File "/usr/local/lib/python3.10/site-packages/whois/_1_query.py", line 171, in _do_whois_query
    p = subprocess.Popen(
  File "/usr/local/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'whois'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dnstwist", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 943, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dnstwist.py", line 1297, in run
    if args.debug:
AttributeError: 'Namespace' object has no attribute 'debug'

[elceef]

You encountered two different and unrelated issues:

• Python whois package is simply a wrapper for the command line tool, and according to FileNotFoundError is either missing in your system or can't be found
• The latest dnstwist version has silly bug which triggers when handling exceptions for whois.query() - I've just fixed it

Please ensure that whois command is present in your system, then pull the most recent code from Github and test it.

[olifre]

@elceef Thanks, I also hit something similar right now in my automated daily use of the Docker images from DockerHub, during which I pass the arguments --whois --tld /opt/dnstwist/dictionaries/common_tlds.dict:

Traceback (most recent call last):
  File "/opt/dnstwist/dnstwist.py", line 1295, in run
    whoisq = whois.query('.'.join([dom, tld]))
  File "/usr/lib/python3/dist-packages/whois/__init__.py", line 54, in query
    raise Exception('Unknown TLD: %s\n(all known TLD: %s)' % (tld, list(TLD_RE.keys())))
Exception: Unknown TLD: ph
(all known TLD: ['com', 'at', 'uk', 'pl', 'be', 'biz', 'br', 'ca', 'co', 'jp', 'co_jp', 'cz', 'de', 'eu', 'fr', 'info', 'io', 'it', 'ru', 'lv', 'me', 'mx', 'name', 'net', 'nz', 'org', 'ru_rf', 'sh', 'us'])

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/dnstwist/dnstwist.py", line 1324, in <module>
    run()
  File "/opt/dnstwist/dnstwist.py", line 943, in wrapper
    result = func(*args, **kwargs)
  File "/opt/dnstwist/dnstwist.py", line 1297, in run
    if args.debug:
AttributeError: 'Namespace' object has no attribute 'debug'

Seeing the last two commits, I presume this was also fixed (i.e. by fixing the bug in exception handling for whois), correct? In that case, it would be great to get fresh Docker images whenever convenient for you. Thanks in advance! :+1:

[domderen]

Hey @elceef, thanks for a quick turnaround! I tried the latest version from master and it worked great (after installing whois 😅).  

[elceef]

Thanks for reporting this issue!

@olifre bug-free Docker images have been pushed!

[olifre]

@elceef Great, many thanks, that was fast! I tested with my usual GitHub actions runs which use the latest containers and they work fine again :+1: .