Too many 429 error responses. Max retries exceeded.

[tahirs95]

I have to run pytrends over 4000 keywords on daily basis on my Google Cloud VM. I read other issues and upgraded version to 4.9.2 as well but still I am getting following errors.

1. Max retries exceeded. urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='trends.google.com', port=443): Max retries exceeded with url: /trends/api/explore?hl=en-US&tz=360&req=%7B%22comparisonItem%22%3A+%5B%7B%22keyword%22%3A+%22google+nest%22%2C+%22time%22%3A+%22today+5-y%22%2C+%22geo%22%3A+%22GB%22%7D%5D%2C+%22category%22%3A+0%2C+%22property%22%3A+%22%22%7D (Caused by ResponseError('too many 429 error responses')) I have purchased residential proxies as well but it still throws error.

2. Empty Dataframes. Sometimes, I get empty data for few of the keywords but I get data for same keyword after sometime, so there is data inconsistency.

For example: Input: keyword: voice speaker country_code: GB timeframe: today 5-y

Output: Empty DataFrame

[Terseus]

Hi @tahirs95 ,

The empty DataFrames are built using the response from Google Trends, who sometimes returns empty responses even for keywords that previously it returned information about; we don't have control over Google Trends data, sorry.

As for the 429 error responses it may be just the expected behavior from Google Trends, how much keywords are you downloading before starting to get the 429 error responses?

[naturallydeer]

Hi, I'm facing this issue after the 4.9.2 update. Tried the request NID cookie suggestion but no luck. The 429 error happens at my first keyword.

Edit: happens at my first keyword for related_topics(). Works ok for related_queries().

[tahirs95]

Hi @tahirs95 ,

The empty DataFrames are built using the response from Google Trends, who sometimes returns empty responses even for keywords that previously it returned information about; we don't have control over Google Trends data, sorry.

As for the 429 error responses it may be just the expected behavior from Google Trends, how much keywords are you downloading before starting to get the 429 error responses?

I am downloading 4000 keywords in a go and I am using residential proxies as well. The script starts throwing 429 after 1000 keywords.

[Terseus]

@tahirs95 I'm sorry but that falls within the expected behavior of the Trends API.

[tahirs95]

@Terseus I am using proxies as well, still I can't use the library over 4000 keywords, What's the recommended way?

[Terseus]

@tahirs95

Sorry for the delay.

Unfortunately there isn't much you can do, your options are:

• use proxies; I recommend to use a provider with a superproxy (most are, some very cheap doesn't).
• retry the requests after 2 hours. -develop a good strategy of downloading only the keywords you cannot the last time.

[Terseus]

@tahirs95 Given that this problem is out of our control, I'm closing this issue.