GetChats is slow

[kigorw]

Hi, I'm using java client implementation as in example.java. Loading 100 chats from cold start takes more than 30 seconds and it's slow for my case. I think the bottleneck with downloading avatars for every requested chat. if it's a right guess, Is there a way not to download files until I need them?

Thanks

[levlam]

None of the files are downloaded automatically by TDLib.

[isopen]

I think you need to revise the rendering algorithm. Json is coming fast (TDLib uses TCP). Maybe you should double-check the timing on the long-poll. Check how fast TDLib establishes a connection.

[JunaidBabu]

I'm not sure if this is related. I have around 800 chats in my current account. When I call getChats with limit 10, it returns the first 10 results, but keeps returning the rest of the 800 chats, which means client_receive will throw updateNewChat, updateUser, updateChatOrder... and so on for all the 800 chats.

A cold start with my account takes more than 15 seconds for this reason. Is there a way to stop the updates from coming. For example when I call getChat(limit:10), return details related to those 10 Chats, nothing more.

[isopen]

Interesting example. At first sight everything is fine. Can you please reset the log to look at it? What if for this case using non-synchronous list updates in the view?

[JunaidBabu]

The problem is not with my UI. Any request sent during getChats process is going on will be delayed,

Consider this scenario: Time T = 0.0 second : call getChats(limit:10..) Time T = 0.5 seconds: client_receive returns all chat related updates (updateChat,updateUser, updateOrder etc) Time T = 1.0 second : client_receive returns the result of 10 chats Time T = 2.0 seconds: client_receive keeps getting the data for the rest of 800 chats Time T = 3.0 seconds: I open a chat. calls getChatHistory() Time T = 4.0 seconds: client_receive still getting details of chats Time T = 10.0 seconds: client_receive still same.. Time T = 15.0 seconds: client_receive done with getChats' after-effects Time T = 16.0 seconds: client_receive returns results of getChatHistory called 13 seconds ago

Here, I could've saved the 13 seconds if getChats did not go ahead pre-fetching all chat information.

I'm sorry, what do you mean by reset the logs?

[vidit-bhatia]

@JunaidBabu I have tried this in android and do not see it the way you explain it here, AFAIK tdlib fetches the data for caching no matter you want 10 or 50 and keeps it in the local database , It will not send updates to the app unless you request the next batch.

[levlam]

@JunaidBabu This is called chat preloading and doesn't affect other requests. Are you sure that you send getChatHistory request on the 3rd second? Could you send TDLib's log with verbosity level 4 to @tdlib_bot?

[levlam]

@kigorw Could you also send TDLib's log with verbosity level 4 to @tdlib_bot to find the cause of the problem?

[kigorw]

I'll try. Will need to figure out how to anonymize data.

On Wed, 25 Sep 2019 at 16:01, Aliaksei Levin notifications@github.com wrote:

@kigorw https://github.com/kigorw Could you also send TDLib's log with verbosity level 4 to @tdlib_bot to find the cause of the problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tdlib/td/issues/707?email_source=notifications&email_token=AACODXXYEFYTKJZSFI3Y4PTQLNHK5A5CNFSM4IZPXIVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7RUMCY#issuecomment-534988299, or mute the thread https://github.com/notifications/unsubscribe-auth/AACODXXZLOX54ZO3DECO53LQLNHK5ANCNFSM4IZPXIVA .

[vidit-bhatia]

@kigorw you do not have to anonymize the data just connect to the test server and generate some test messages .

https://core.telegram.org/tdlib/docs/classtd_1_1td__api_1_1tdlib_parameters.html if you start your client with use_testdc is true it will connect to the test telegram dc

[JunaidBabu]

I'm sorry, it was problem with my implementation.

I have been calling client_receive and client_send in the same thread. And client_receive is called in a while loop which breaks only when there is no update. So the getChatHistory was actually called at the end of 15th second. Sorry for the confusion.

[isopen]

You can try using shells through the json interface. Many of the shells are already created by the community.

[levlam]

@JunaidBabu Thanks for the confirmation.

Call client_receive and client_send from the same thread is perfectly fine, but you need to always call client_receive with timeout 0 and sleep or wait for user input from time to time to not exhaust 100% CPU. This is exactly what we do in our C++ example: https://github.com/tdlib/td/blob/a2429d595ce38456240717219fe4e61c3ba8b38a/example/cpp/td_example.cpp#L83

[kigorw]

@levlam I've sent logs_anon.txt.zip to @tdlib_bot. It reproduces an issue - it's even slower probably because of log verbosity.

I also tried to remove all sensitive data from it (I hope it will not prevent you to understand a root cause)

the only reason I can think of right now is outdated (saved after login)td.binlog which I reuse between service restarts

[levlam]

@kigorw The problem happens exactly because you try to use very old database state restored from the backup. If you want to use it in such way then you must disable secret chats and all databases in setTdlibParameters and set option "ignore_background_updates" to true before TDLib is initialized.

[kigorw]

@levlam thanks, it helped!