search

Jefferson-Henrique / GetOldTweets-python

A project written in Python to get old tweets, it bypass some limitations of Twitter Official API.
MIT License
1.35k stars 809 forks source link

limit of crawling #258

Open adamrag opened 4 years ago

adamrag commented 4 years ago

please what is the limit of crawling using advanced search,when we do we need to use proxy to avoid block our request from twitter? thanks.

JayMehtaUK commented 4 years ago

What I do is to surround (tweets = got.manager.TweetManager.getTweets(tweetCriteria)) in a try except. And if you get blocked then make your program sleep for a bit and retry getting the tweets again. It will take longer but at least your program wont stop if you get blocked and can retry at a specified interval (I did 30 seconds).

Alternatively you can use a proxy as shown in this commit: https://github.com/Jefferson-Henrique/GetOldTweets-python/pull/49