search

GeneralMills / pytrends

Pseudo API for Google Trends
Other
3.25k stars 815 forks source link

Connect Timeout issue with proxies #419

Open rahulramaswamy opened 4 years ago

rahulramaswamy commented 4 years ago

I'm trying to use proxies from this url: https://sslproxies.org/ to pull large amounts of data from google trends, but I'm getting an issue in this line of code: pytrend = TrendReq(hl='en-US', tz=360, timeout=(10,25), proxies=list(proxies), retries=2, backoff_factor=0.1, requests_args={'verify':False}) the list of proxies is from scraping the above website. This is the issue: ConnectTimeout: HTTPSConnectionPool(host='trends.google.com', port=443): Max retries exceeded with url: /?geo=US (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001E16F5FB4C8>, 'Connection to 118.174.232.44 timed out. (connect timeout=10)')) Please advise.

rahulramaswamy commented 4 years ago

How can I get proxies that are valid and work with pytrends?

Kkkassini commented 4 years ago

problem is that you are making "too many" requests in the same time, add some try-except + sleep machanism maybe...

Cursed-Gato commented 2 years ago

Alright, the issue can be two things. One you don't have working proxies, you can fix this by using a proxy checker like https://hidemy.name/en/proxy-checker/. The second one is that for some reason the person that programmed this assumed you have to use HTTPS and for some reason, it wont work properly. So, to fix this, go to request.py for the library, and change all the instances from HTTPS to HTTP, that is don't change any that are full URIs. after that make sure that in the proxy list you have https:// and not http://

alexpn98 commented 1 year ago

Hello @rahulramaswamy, did any of the solutions work ? I have the same problem.

Cursed-Gato commented 1 year ago

Hello @rahulramaswamy, did any of the solutions work ? I have the same problem.

Yea just put the procies as http instrad of https and it will work perfectly fine