aio_whois_domain() running infinitely

pogzyb / asyncwhois

Python WHOIS and RDAP utility for querying and parsing information about Domains, IPv4s, IPv6s, and AS numbers

MIT License

63 stars 18 forks source link

aio_whois_domain() running infinitely #36

Closed masquerad3r closed 1 year ago

masquerad3r commented 2 years ago

While using the library to perform mass WHOIS lookups, I found that the aio_whois_domain() function is not respecting the timeout parameter and running the query infinitely.

Example query performed:

loop = asyncio.get_event_loop()
result = loop.run_until_complete(asyncwhois.aio_whois_domain("1stchoicebankca.com"))

As per the source code, default timeout is set to 10 seconds, but surely its taking more than that.

pogzyb commented 2 years ago

Hi Prerak,

Thanks for posting. I briefly looked into this on my end and things seem to be good as far as respecting the timeout parameter: atimeout With the domain you mentioned: atimeout2 I will dig a little deeper though to be sure. In the meantime, here a few questions that can help me out:

Does the query still return; it just seems to be taking longer than the timeout value? or does it actually hang infinitely?
Are you using a proxy?
Have you ruled out the possibility that is it your internet connection?
When you say "mass lookups", are you using anything special to run multiple Coroutines? e.g. asyncio.gather(*list_of_whois_lookups)

masquerad3r commented 2 years ago

Hi pogzyb,

Please find the answers to your query below:

Does the query still return; it just seems to be taking longer than the timeout value? or does it actually hang infinitely? No, the query does not return, it just hangs indefinitely. I waited more than 5 minutes for the query to return a value. Anyhow, it should respect the configured timeout.
Are you using a proxy? Yes, a proxy is being used. Without it, I am also not facing any issues. But the environment in which this needs to be executed requires a proxy connection.
Have you ruled out the possibility that is it your internet connection? Yes, my internet connectivity is not a problem. Apart from a handful of these domains (as I mentioned before), all others are giving results in the expected time.
When you say "mass lookups", are you using anything special to run multiple Coroutines? Yes, asyncio.gather() is being used.

pogzyb commented 2 years ago

@masquerad3r Thanks for the info!

I don't have a great answer right now, but if your query is hanging infinitely, then this issue may be related: #34. PR #35 was created and merged to address this for .tk domains, but hasn't yet been included in a release, so you'll need to pip install from the master branch.

It's possible there are other TLD parsers that are causing the same issue. If you run into this issue again please send me a domain that's causing the issue and I will look into it thoroughly.

pogzyb commented 2 years ago

@masquerad3r v1.0.1 is now available and contains the fix for .tk domains, which may have been the underlying issue. I'll keep this open in case the problem persists and/or other issues around "infinitely running" queries/parsing are identified.

Thanks!