elceef / dnstwist

Domain name permutation engine for detecting homograph phishing attacks, typo squatting, and brand impersonation
https://dnstwist.it
Apache License 2.0
4.85k stars 767 forks source link

Docker image is missing whois binary - resulting in missing whois_created date #200

Closed xapax closed 1 year ago

xapax commented 1 year ago

Hey! Thanks for a great project!

I have noticed that when I run the docker container with the --whois flag dnstwist is not always returning whois_created date.

It appears that the pip package whois requires the OS whois (/usr/bin/whois) to be installed in order to be able to retrieve the whois-create date.

This issue can be solved by updating the Dockerfile to install whois.

# docker build -t dnstwist .
# docker build -t dnstwist:phash --build-arg phash=1 .
# [..]
# Here
RUN apt-get update && \
export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y --no-install-recommends whois python3-dnspython python3-tld python3-geoip python3-whois ca-certificates && 
# [...]

Also, I noticed that when running the docker container the python3-whois package is installed from Debian. The Debian package python3-whois is using version 0.8, which was released in April 2019. I guess this means that the docker image and the pip version of dnstwist installs different versions of packages. This might cause some confusion or introduce inconsistencies between the docker version and the pip-version.

In the docker image it looks like version 0.8 is installed:

➜   docker run -it --entrypoint /bin/sh elceef/dnstwist       
# cat /usr/lib/python3/dist-packages/whois-0.8.egg-info
Metadata-Version: 1.1
Name: whois
Version: 0.8
Summary: Python package for retrieving WHOIS information of domains.
Home-page: https://github.com/DannyCork/python-whois/
Author: DannyCork
Author-email: ddarko@ddarko.org
License: MIT http://www.opensource.org/licenses/mit-license.php
Download-URL: https://github.com/DannyCork/python-whois/releases/tag/0.8

While installing from pip version 0.9.27 of whois is installed.

Anyways, thanks again for a great project!

elceef commented 1 year ago

Thanks for reporting this. Very recently I've decided to finally abandon python3-whois package (along with its PyPI counterpart) which, as you noticed yourself, is a wrapper for the whois command line utility. Yesterday I replaced it with my own pure-Python implementation which also turned out to be significantly faster. I've just updated the Dockerfile so I would appreciate if you could verify that it does solve the issue.

xapax commented 1 year ago

Wow! That was a fast respons! I tried it out, and it works great! Thanks!