search

digitalmethodsinitiative / dmi-tcat

Digital Methods Initiative - Twitter Capture and Analysis Toolset
Apache License 2.0
367 stars 114 forks source link

Debugging rate limiting issue #313

Closed cwalck closed 6 years ago

cwalck commented 6 years ago

Hey all,

How would one go about debugging a rate limiting issue? Despite having 8 API keys in my config.php, I'm still getting rate limited a few times a day. I'm not convinced keys 2-7 are actually being used.

Could I write the current $twitter_consumer_key to log file? If so, where might I insert the logit() statement?

ErikBorra commented 6 years ago

The extra keys only help to advance more quickly through REST API calls (such as search and lookup).

The only solution to avoid hitting rate limits with the streaming API is to set up a new machine with another IP address and another API key. Move some of the bins with keywords that hit a lot of tweets there. Be mindful of Twitter's TOS.