fa0311 / TwitterInternalAPIDocument

Twitter Internal API Document
Other
385 stars 37 forks source link

Timelines eventually stop returning posts #550

Open catdevnull opened 5 months ago

catdevnull commented 5 months ago

Hey! I'm doing some Twitter research and I'm stumbling upon an issue that I'm not sure if there's a way to solve. One of the things that I want to scrap is all of the liked tweets of certain accounts. However, at some point (~5k tweets) the timelines start returning no tweets, with the bottom cursor being equal to the one that's being requested. Is there a way to bypass this? Also, is there possibly some kind of Discord/Forum community to discuss these sort of Twitter internal API stuff? Thanks!

fa0311 commented 5 months ago

hi @catdevnull.

One of the things that I want to scrap is all of the liked tweets of certain accounts. However, at some point (~5k tweets) the timelines start returning no tweets, with the bottom cursor being equal to the one that's being requested.

Unfortunately, this is a Twitter specification and there is no solution. Twitter has a complex DB, and for optimization purposes, old likes will no longer be referenced by the account.

Also, is there possibly some kind of Discord/Forum community to discuss these sort of Twitter internal API stuff?

I do not know it. If a forum exists I would like to know about it too.

A forum exists that I administer, but only I use it :( Ask here and I will answer.

https://github.com/fa0311/twitter-openapi/discussions