search

MartinKBeck / TwitterScraper

Repository containing all files relevant to my basic and advanced tweet scraping articles.
196 stars 117 forks source link

Limit of quantity of tweets with snscrape #1

Closed JSMboli closed 3 years ago

JSMboli commented 3 years ago

Hi Martin Thank you for the good work you are doing.

I was wondering if there is a limit to the number of tweets one can scrape with snscrape. What about the date, any limit? Thank you.

MartinBeckUT commented 3 years ago

Hi, I appreciate the sentiment.

From my understanding, there's no hard limit imposed by snscrape. Any limit you'll come across are probably from IP banning. Speaking from personal experience I scraped 70k tweets in a day and was fine. You shouldn't be banned unless you're doing some crazy stuff like multiple scraping on the same IP pulling in 100s of thousands of tweets in a day.

In regards to date, there should be no limit all the way up until 2006, but that's a limit imposed by Twitter. Otherwise, you should be fine searching whenever you need to.