When creating an instance of Client (found in twitter_ads.client) you can pass a dict with additional options, using the options parameter. One of this options is handle_rate_limit. When passed, the library is supposed to look for the rate limit headers returned by the API and sleep the execution when such a limit is encountered. However, it would appear that the wait time is insufficient, as the next request after the wait period fails with a rate limit error.
The above code is ran for an export range of one day. It is repeated for multiple accounts. This is the output when a rate limit is reached:
time=2022-07-26 17:22:50 name=twitter_ads.http level=WARNING message=Request reached Rate Limit: resume in 592 seconds
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "*********/utils/backfill.py", line 110, in run_backfill
for entity_data in results:
File "*********/utils/backfill.py", line 74, in _download_data
analytics_data: List[dict] = Campaign.all_stats(
File "*********/twitter-ads-env/lib/python3.8/site-packages/twitter_ads/analytics.py", line 78, in all_stats
response = Request(account.client, 'get', resource, params=params).perform()
File "*********/twitter-ads-env/lib/python3.8/site-packages/twitter_ads/http.py", line 70, in perform
raise Error.from_response(response)
twitter_ads.error.RateLimit: <RateLimit object at 0x7f2b79a75520 code=429 details=None>
Expected behaviour:
The library should wait the appropriate amount of time before attempting to call the API again to completely avoid RateLimit errors.
Observed behavior:
At the current state, the handle_rate_limit option serves no purpose. Despite waiting a certain amount of time, the rate limit error is encountered anyway.
Description:
When creating an instance of
Client
(found intwitter_ads.client
) you can pass a dict with additional options, using theoptions
parameter. One of this options ishandle_rate_limit
. When passed, the library is supposed to look for the rate limit headers returned by the API and sleep the execution when such a limit is encountered. However, it would appear that the wait time is insufficient, as the next request after the wait period fails with a rate limit error.How to replicate:
The above code is ran for an export range of one day. It is repeated for multiple accounts. This is the output when a rate limit is reached:
Expected behaviour:
The library should wait the appropriate amount of time before attempting to call the API again to completely avoid
RateLimit
errors.Observed behavior:
At the current state, the
handle_rate_limit
option serves no purpose. Despite waiting a certain amount of time, the rate limit error is encountered anyway.