Limit reached and ` too many requests` problem.

[nkint]

Hi, I'm keeping reaching too many requests after some test. Is there a documentation about limitations? How many request can I do?

[pat310]

Hi @nkint! Google does have a request limit which; unfortunately, I don't have numbers for. People have used proxy servers to distribute their requests over multiple IP's with success.

I tried implementing a login functionality similar to pytrends but I found that it did not make a difference with the allowable number of requests and I got scared about having people pass their google passwords in 😨so I abandoned that pursuit. One thing I have been thinking about but haven't had the time to try yet is to use something like puppeteer ¯_(ツ)_/¯

[mikehern]

Hello - I'm getting a 429 error but haven't hit the API from this IP address in the past 2 days.

It looks like the main trends website did get an update. Is there any chance the endpoint just needs to be updated?

I also found this commit with an unofficial estimate of the rate limit.

[pat310]

Yea that's likely. I'll try to take a look and get it fixed!

On May 28, 2018, at 20:25, Mike Hernandez notifications@github.com wrote:

Hello - I'm getting a 429 error but haven't hit the API from this IP address in the past 2 days.

It looks like the main trends website did get an update. Is there any chance the endpoint just needs to be updated?

I also found this commit with an unofficial estimate of the rate limit.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[nautonk]

i get this problem too, i try to change user-agent but it seem like it does not work. Btw it works good if i use aria2c (limit not reached).

Thank you so much, very good repo :)

[splatcollision]

Also getting this problem.

Following up on a related issue in a python trends library, does it make sense to try and detect the statusCode === 429 and presence of a 'set-cookie' header, and automatically resending the request with the correct cookie?

https://github.com/GeneralMills/pytrends/issues/243#issuecomment-392778958 For reference...

[zachokeeffe]

I got it to work in R: https://github.com/PMassicotte/gtrendsR/issues/273.

Not sure how to implement it in JavaScript, but it's apparently the "OTZ" cookie from apis.google.com that you need set.

[pat310]

Should be corrected in version 4.6.2

[nautonk]

Latest version works good. Thank you so much :)

[anassis]

I ahve found that it is more related to scrapping. In R Studio, I just uncheck "Use Internet Explorer library/proxy for HTTP" at Packages, under Options, and it seens solved the problem.