x-fran / g-trends

Google Trends API for PHP
https://tarlabs.es
MIT License
112 stars 47 forks source link

How to avoid HTTP status 429 (too many requests)? #56

Open Rombersoft opened 1 year ago

Rombersoft commented 1 year ago

Who can share experiences about problems with HttpStatus 429 (Too many requests)? I noticed that the developer thought about using the proxy server. Does it exist some statistics about what I must do in order to be banned by Google Corporation? Public API is missing and how much is legal to use google back end in order to show google trends results on my own site???

nekromoff commented 1 year ago

change lines: 141 to: throw new \Exception('Maximum five keywords accepted. Split and loop, if more'); // optional 276 to: throw new \Exception($client->getResponse()->getReasonPhrase().' / '.$client->getResponse()->getStatusCode());

then use sleep($delay) with loop increasing sleep time $delay e.g. by 5s each time, when exception is thrown until you get the trend results.

example:

$result = [];
$sleep = 0;
while ($result == []) {
    try {
        $result = $gt->getInterestOverTime($keyword_group);
    } catch (\Exception $e) {
        $sleep = $sleep + 5;
    }
    sleep($sleep);
}
jabeztadesse commented 1 month ago

Facing the same issue ...

nekromoff commented 1 month ago

My solution does not work anymore. Google has clamped down on scraping Trends. You might want to use some commercial data provider that has robust scraping mechanisms.

jabeztadesse commented 1 month ago

Google uses more advanced headers and cookies now, right?

nekromoff commented 1 month ago

Google uses more advanced headers and cookies now, right?

More advanced scraper detection.