[Twitter] Lowest rate limit to avoid account lock/captcha/401 unauthorized

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites

GNU General Public License v2.0

11.96k stars 975 forks source link

[Twitter] Lowest rate limit to avoid account lock/captcha/401 unauthorized #5330

Open a84r7a3rga76fg opened 8 months ago

a84r7a3rga76fg commented 8 months ago

Has anyone figured out what the lowest rate limit is?

mikf commented 8 months ago

This depends on a lot of factors, I think.

API endpoint
geo location
overall Twitter use and abuse by other users
date
moon phase
...

For me, rate limits are currently at

95/15m UserByScreenName
500/15m UserMedia
50/15m UserTweets
50/15m SearchTimeline

By default, gallery-dl uses UserByScreenName once per user, followed by UserMedia, followed by SearchTimeline.

trip54654 commented 8 months ago

I guess getting your account(s) locked and having to solve captcha every once in a while is just something you have to live with.

What I find more annoying is that after a while, my account/IP gets permanently rate limited for at least 12 hours. Switching to another account helps sometimes. I'm thinking about rotating accounts. Currently they're easy to create, it accepted the first temp mail I gave it, and doesn't want a mobile phone number.

Has anyone figured out the ideal way to scrape twitter?

a84r7a3rga76fg commented 8 months ago

I guess getting your account(s) locked and having to solve captcha every once in a while is just something you have to live with.

This would've been fine with me if there was an option to resume the download process.