search

joepie91 / python-whois

A python module for retrieving and parsing WHOIS data
Do What The F*ck You Want To Public License
399 stars 187 forks source link

Problem with referral_url #116

Open mariuskagc opened 8 years ago

mariuskagc commented 8 years ago

Hi Joe, I think I found the cause of an issue in your module (it may be related with https://github.com/joepie91/python-whois/issues/76 ): I am using python 2.7 with pythonwhois v2.4.3 and I was having problems with some .org (cofteruel.org, apache.org, bancodesangreanimal.org...) TLD (that's what I found so far):

from pythonwhois import get_whois whois_data = get_whois("apache.org") Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/pythonwhois/init.py", line 4, in get_whois raw_data, server_list = net.get_whois_raw(domain, with_server_list=True) File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 70, in get_whois_raw return get_whois_raw(domain, referal_server, new_list, server_list=server_list, with_server_list=with_server_list) File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 45, in get_whois_raw response = whois_request(request_domain, target_server) File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 89, in whois_request sock.connect((server, port)) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused

So, I tried to look for the problem, I debugged it and use another library that actually works to compare them: With the other library I get:

'updated_date': datetime.datetime(2015, 8, 11, 21, 32, 3), u'status': [u'clientDeleteProhibited https://www.icann.org/epp#clientDeleteProhibited', u'clientTransferProhibited https://www.icann.org/epp#clientTransferProhibited', u'clientUpdateProhibited https://www.icann.org/epp#clientUpdateProhibited'], u'name': u'Host Master', u'dnssec': u'unsigned', u'city': u'Forest Hill', u'expiration_date': datetime.datetime(2021, 4, 12, 4, 0), u'zipcode': u'21050-2747', u'domain_name': u'APACHE.ORG', u'country': u'US', u'whois_server': None, u'state': u'MD', u'registrar': u'Domain.com, LLC', u'referral_url': u'www.domain.com', u'address': u'1901 Munsey Drive', u'name_servers': [u'NS2.SURFNET.NL', u'NS3.NO-IP.COM', u'NS2.NO-IP.COM', u'NS1.NO-IP.COM', u'NS4.NO-IP.COM'], u'org': u'Apache Software Foundation', u'creation_date': datetime.datetime(1995, 4, 11, 4, 0), u'emails': u'hostmaster-2005-alpha@apache.org'

So I thought that the problem was on the server call. I printed the server with pprint and I got this:

u'whois.iana.org' u'whois.pir.org' u'www.domain.com'

I checked the output of the other library and figured out that problem is in the referral_url. In this case 'referral_url': 'www.domain.com'. Some people fill referral_url with domains that are not a whois server itself and it is altering the behaviour of your module. Also: not always is filled out with www.domain.com. E.g bancodesangreanimal.org has filled out with 'referral_url': 'www.namesilo.com'. I commented that lines (68 and 70) at net.py and it works, but I know that it is not a solution because referral_url should give (as I understood, correct me if I am wrong) more information than the previous (whois.pir.org here)

Besides, I would like to ask you to add a settimeout before the call to the whois server, to get a socket exception if there is no answer from the server instead of waiting for the default timeout. As I needed it I added 10 seconds.

Hope this is helpful to you. Best regards.