search

RipMeApp / ripme

Downloads albums in bulk
MIT License
3.71k stars 629 forks source link

Twitter: Failed to load rate_limit_status.json?resources=statuses #1966

Open nexusofconcordia opened 2 years ago

nexusofconcordia commented 2 years ago

Expected Behavior

RipMe should start crawling the Twitter user's post history for things to download.

Actual Behavior

Crawling immediately fails with the error:

"Failed to load https://api.twitter.com/1.1/application/rate_limit_status.json?resources=statuses after 1 attempts"

There were news recently that Twitter will start enforcing stricter rate limits on the API, so this is probably that. fxtwitter managed to get around this problem somehow, so this might be solvable.

nexusofconcordia commented 2 years ago

Update - having retried it every hour or so, it finally succeeded in scraping without an error. Will keep trying with other Twitter profiles to see if there's consistency to it.

I also just remembered that I've had unusual API rate limit problems before, but those threw an 'API limit exceeded' error instead of the one above, and weren't as persistent(usually going away after a few minutes instead of hours). This could be the same issue, but with RipMe not recognizing it as such due to API changes.