python-twitter-tools / twitter

Python Twitter API
http://mike.verdone.ca/twitter/
MIT License
3.17k stars 710 forks source link

twitter-archiver only works with recent tweets #447

Open msw9 opened 2 years ago

msw9 commented 2 years ago

hello, as said in the title i would like to archive all the tweets of an user even if they've been active for years and, then, have posted thousands of tweets. however, twitter-archiver allows me to collect only a couple of tweets throughout the current year - or sometimes end of last year.

why is there a limit? and how can i bypass it, please?

RouxRC commented 2 years ago

You can't with the API, this is a Twitter limit as documented here: "This method can only return up to 3,200 of a user's most recent Tweets. Native retweets of other statuses by the user is included in this total, regardless of whether include_rts is set to false when requesting this resource."

If you want to go further you can use some scraping tools such as minet for instance like this: minet twitter scrape tweets "from:@YOURTWITTERACCOUNT

Although this won't return you all retweets from the desired account, only originating tweets.

To get everything I think the only solution requires to use the search all route of the API V2 but it is limited to academic accounts which you would need to apply for.