search

epfl-dlab / GoogleTrendsAnchorBank

Google Trends, made easy.
MIT License
101 stars 22 forks source link

Two different behaviors when quota is reached #18

Open hravkin opened 3 years ago

hravkin commented 3 years ago

Hi, Thanks again for the wonderful package and the prompt help in the past.

I've been setting up several anchorbanks in a loop. There are two separate behaviors when my IP reaches the query quota:

1) "Quota reached! Please change IP and press any key to continue" This seems to be the standard message which your code generates. When this comes up I can indeed just change the IP and continue from there.

2) The second option, I pasted the last few normal lines as well as the error message below: (I was hoping there is something I can do to prevent this, as it seems like this way the progress on the anchorbank being built at the time is lost.)

Bad keyword '/m/013v0m', because The request failed: Google returned a response with code 400. 13%|█▎ | 74/568 [03:28<20:41, 2.51s/it] Bad keyword '/m/02nm6c', because The request failed: Google returned a response with code 400. 14%|█▎ | 77/568 [03:36<19:43, 2.41s/it] Bad keyword '/m/084cz', because The request failed: Google returned a response with code 400. 74%|███████▍ | 423/568 [3:41:14<1:15:50, 31.38s/it] Bad keyword '/m/0d5nn', because The request failed: Google returned a response with code 429.

Traceback (most recent call last):

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 139, in _check_keyword rez = self._query_google(keywords=keyword)

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 129, in _query_google self.pytrends.build_payload(kw_list=keywords, **self.CONFIG['PYTRENDS'])

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\pytrends\request.py", line 169, in build_payload self._tokens()

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\pytrends\request.py", line 179, in _tokens trim_chars=4,

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\pytrends\request.py", line 148, in _get_data response=response)

ResponseError: The request failed: Google returned a response with code 429.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "C:\Users\Hersh\Documents\RSV PROJECT\querying_code_updated.py", line 78, in countries_search_data = get_searchdata('/m/02f84', iso_countries, "2021-05-05 2021-10-30")

File "C:\Users\Hersh\Documents\RSV PROJECT\querying_code_updated.py", line 60, in get_search_data t.create_anchorbank() # takes a while to run since it queries Google Trends.

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 799, in create_anchorbank google_results = self._get_google_results()

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 300, in _get_google_results keywords = [k for k in tqdm(keywords, total=len(keywords)) if self._check_keyword(k)]

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 300, in keywords = [k for k in tqdm(keywords, total=len(keywords)) if self._check_keyword(k)]

File "C:\Users\Hersh\Anaconda3\envs\VPN_SWITCHING\lib\site-packages\gtab\core.py", line 147, in _check_keyword raise ConnectionError("Code 429: Query limit reached on this IP!")

ConnectionError: Code 429: Query limit reached on this IP!

ruptho commented 2 years ago

Hi,

I've also encountered this issue with Code 429.