search

TwidereProject / Twidere-Android

http://twidere.com
2.75k stars 377 forks source link

Thread are not working again (v4.1.3) #1327

Open clavelm opened 4 years ago

clavelm commented 4 years ago

See closed issues for v4.1.0

1272

1289

Same problem again for v4.1.3

(sorry I didn't put all the info on that new issue)

Tlaster commented 4 years ago

Could you please post some tweet's link here that has the issue in order to resolve the issue? Seems like it's working here.

clavelm commented 4 years ago

Sorry, I can't find one at the moment. It's threads where there is a lot of retweets with comments on the tweets of the thread and the rest of the thread is missing. I will update this issue as soon as I find one.

Tlaster commented 4 years ago

Ok, I think I know what's happening. As I mentioned in #1272, tweet thread was fetched from a web page of twitter as replies and can show up to 10 replies. it seems like there'are more than 10 replies in the thread. Nitter is an option but I think it's better to fetch data from twitter rather than other services because other services might come with ads or other data that we don't need.

clavelm commented 4 years ago

An example with a thread with 9 messages, a first one (numbered 0 for this post) then 8 numbered from 1 to 8 : When loading the 8th : https://twitter.com/fs0c131y/status/1270274935764680704 I can see the 1~7 former before the message (the 0 is missing, I should be seeing 8 messages, not only 7 but it may be another problem).

When loading the 7th : https://twitter.com/fs0c131y/status/1270267441361362945 I can see the 0~6 messages before, but there is 3 responses from other people before seeing the 8th message of the thread. Screenshot_20200610-165301_Twidere Screenshot_20200610-165314_Twidere

When loading the 0 : https://twitter.com/fs0c131y/status/1270135790220312577 the 1~8 messages are not visible. I only see 10 responses from other people and that's all, I can't load more than 10 responses. I must add there is a delay of 8 hours between message 0 and message 1 of the thread, so other people had time to respond to that first message before the rest of the thread followed.

Tlaster commented 4 years ago

Thanks, now I know what's happening.

but there is 3 responses from other people before seeing the 8th message of the thread.

I only see 10 responses from other people and that's all, I can't load more than 10 responses.

I think these are the issue that related to Twidere, others are Twitter's issues I think, I can reproduce then on Twitter's official client, or am I missing something?

I think we can add an option to switch to nitter in order to fetch all replies and threads, currently, we can only load up to 10 replies in some tweets since Twidere is fetching replies from Twitter's legacy web page.

Xefir commented 4 years ago

Just out of curiosity, why Twidere does not fetch replies and thread from API instead of curling Twitter website ?

Tlaster commented 4 years ago

I don't see any API for Twitter to fetch tweet reply and thread, or am I missing something?

Xefir commented 4 years ago

You're right, I though that there was an API for that but seems like there's nothing

clavelm commented 4 years ago

I'll check using Twitter in my browser, but I think they load threads as it should, responses from the same user are priorized. I don't know how Twidere gets the tweets from Twitter, api or scraping of the web site (as NewPipe do for YouTube) but if it's the latter, it should be possible to emulate the same behavior than the Twitter website, shouldn't it?

clavelm commented 4 years ago

Thanks, now I know what's happening.

but there is 3 responses from other people before seeing the 8th message of the thread.

I only see 10 responses from other people and that's all, I can't load more than 10 responses.

I think these are the issue that related to Twidere, others are Twitter's issues I think, I can reproduce then on Twitter's official client, or am I missing something?

I think we can add an option to switch to nitter in order to fetch all replies and threads, currently, we can only load up to 10 replies in some tweets since Twidere is fetching replies from Twitter's legacy web page.

What I can say using mobile.twitter.com in Firefox for Android.

When loading the 8th message, 1~7 are displayed before, the 0th is missing, so it's not a Twidere problem.

When loading the 7th, the 1~6 are displayed before, the 0th is still missing (while Twidere displays it) so I think Twitter ignores it because of the time discrepancy between the 0th and the 1st (8h). The 8th message is displayed just after the 7th.

When loading the 0th, the thread is not displayed, it's consistent that this message is not part of the thread for Twitter.

When loading one of the the 1~8 messages, there is a title at the top of the Twitter pages : Discussion (French for thread).

When loading the last message of a longer thread in Twitter, all the previous messages from the thread are loaded. Same with Twidere (see French thread https://twitter.com/reesmarc/status/1270558042162114562?s=20 for example). So the missing 0th thread is not a Twidere problem. What is strange is why it's missing when displaying the 8th message, but present when loading the 7th.

So Twitter identifies and assembles all the messages from threads (messages responding to the previous one of the same user, in a specific time span for all messages or a max time between messages), but there is no API for it, that's what you are saying?

So no flag or thread id gotten with a message to know it's part of a thread and no API request with a thread id to get all and only the thread messages.

The only way is to scrap the website or use API from other sources. They really want users to use the official site / client for the ads I think.

Tlaster commented 4 years ago

So Twitter identifies and assembles all the messages from threads (messages responding to the previous one of the same user, in a specific time span for all messages or a max time between messages), but there is no API for it, that's what you are saying?

Yes, I can not find the API to fetching reply and thread. if there is please let me know.

So no flag or thread id gotten with a message to know it's part of a thread and no API request with a thread id to get all and only the thread messages.

There is a flag called "in_reply_to_status_id" from Twitter API, but it's not a "thread id". Twidere uses that to load the thread's tweets by getting each tweet recursively.

Rudloff commented 4 years ago

There is a feature request for this: https://twitterdevfeedback.uservoice.com/forums/921790-twitter-developer-labs/suggestions/37629283-build-an-api-to-create-and-retrieve-tweet-threads