vladkens / twscrape

2024! Twitter API scrapper with authorization support. Allows you to scrape search results, User's profiles (followers/following), Tweets (favoriters/retweeters) and more.
https://pypi.org/project/twscrape/
MIT License
784 stars 102 forks source link

cannot get more than 142 tweets from api.user_tweets #166

Closed jt6677 closed 2 months ago

jt6677 commented 2 months ago

I have 5 accounts logged in. api = API('accounts.db') tweets = await gather(api.user_tweets(user_id, limit=2000))

I could only get 142 tweets back every time before getting twscrape.queue_client:_check_rep:173 - Dependency error (request skipped): (131) Dependency: Internal error. (131)

what should I do? Thank you

vladkens commented 2 months ago

It's hard to tell what the problem is. It's an unrecognized Twitter bug at the moment.

If you have managed to reproduce this error consistently - please share the request - this way it will be possible to investigate the reason.

jt6677 commented 2 months ago

Using exact code, api = API('accounts.db') tweets = await gather(api.user_tweets(user_id, limit=2000)) I tried 197785542 and could not get more than 150 tweet, and error out with 131.(ten tries) HOWEVER, I tried 158439326 and I got 690. Interesting....

vladkens commented 2 months ago

@jt6677 thank you. That was helpful. Will be released in v0.12 in few days.

With dependency error API also returned data as expected - response validation improved.