search

samwize / python-email-crawler

Search on Google, and crawls for emails related to the result
292 stars 127 forks source link

It keeps searching forever. #24

Open ghost opened 6 years ago

ghost commented 6 years ago

It just searches everywhere... I sometimes think that it searches every single website.

Nothing is wrong, it just does not stop, I can see the websites it is searching in, but after 1 - 1.30 hours later it keeps searching unrelated website and does not stop searching.

Are my keywords wrong? What can I do?

kopevgale commented 6 years ago

I will join the question

lonalore commented 2 years ago

I run into tha same issue. I replaced the following row in email_crawler.py file (line 95):

request = urllib2.urlopen(req)

with this:

request = urllib2.urlopen(req, timeout=10)