[Brainstorming] Performances

[rpadovani]

Hello, first of all, thanks for your awesome work! Thanks to you I have Telegram on my Ubuntu Phone, and that is amazing! Thanks so much!

I'm opening this bug to talk about how improving performances of the library, focusing on database queries and APIs calls.

I spent some days investigating them and I would like to share with you what I've found. I would love your opinions on this.

I would like to keep the discussion here on high-level, because my QT/C++ skills are low and because I think problem is more about logic behing some APIs calls that in the code itself.

I'm going to talk about different sides of the problem: I did not open different bugs because I think we need a general vision before splitting in various subtask.

App startup

Current

At the startup the app calls TelegramDialogsModel::recheck(), which calls p->telegram->messagesGetDialogs(0,0,1000);. This is an API request which returns a list of chats, a list of last messages from each chat, a list of groups mentioned in the chats and a list of users mentioned in messages and groups.

The reply is managed by TelegramQml::messagesGetDialogs_slt() which takes the reply and for each object in the reply (at moment in my account 85 users, 28 chats, 100 messages and 100 dialogs) do an INSERT query in the database. Plus it removes old dialogs so there are more queries.

This is a waste of I/O, because if it is not the first start the user already has on his phone the majority of these data.

How to improve

TelegramDialogsModel::recheck() function should first of all take all the data from cache / database, check what's the highest dialog id it has and then call (let's call it maxId) and then call p->telegram->messagesGetDialogs(maxId,0,1000);.

In this way the server replies only with new created dialogs (that most of times will be 0).

Of course the app needs also to find new messages in chats that already exists.

There are two possible ways here:

First solution, it requires more bandwidth but less CPU time

We take all the conversations id we have in the database (peer:InputPeer) and for each we call messages.getHistory passing the number of messages we have as offset. In that way server replies only with new messages, if any, but it requires a lot of API calls.

Second solution, 1 API call but more CPU time

We take all peer:InputPeer and we create a vector for messages.getMessages. When we read the reply we check the id of the message, and if is bigger than the top message id of the conversation we have we save it, otherwise we drop it.

I know my english is bad, so let's say with pseudocode:

Q_FOREACH(m, incomingMessages) {
  if (dialogs.value(m.toId)->topMessage() < m.id()) {
    database->saveMessage(m);
  }
}

Reconnecting to network

Current

There are two events (authLoggedInChanged() and connectedChanged()) which call the function TelegramDialogsModel::recheck().

How to improve

If we rewrite TelegramDialogsModel::recheck() as said before these events will improve as well. We just need to check authLoggedInChanged() drops elements from prevoius account.

Opening a chat

Current

When a chat is opened TelegramMessagesModel::loadMore() is called, which calls tgObject->messagesGetHistory(peer, p->load_count, p->maxId, p->load_limit);

This is great, because the second argument of messagesGetHistory is the offset, but unfortunately p->load_count is always 0, so the app APIs will always return last 50 messages, and all 50 will be written to the database again.

How to improve

I haven't investigated this too deeply, but I think is because each time a dialog is selected a new istance of the class is created, so load_count is set to 0.

We need to create a method to know how many messages we already have in a dialog and pass it to tgObject->messagesGetHistory, so we take only last messages.

Receiving a message

I haven't found the root cause, but when a message arrives, TelegramQml::timerEvent is called, which cause p->telegram->messagesGetDialogs(0,0,1000); to be invoked. As explained before, this is real bad. We need to understand here if timerEvent() is needed and, if it is, if we want to messagesGetDialogs(). If yes we can at least set an offset, otherwise we can find a more suitable function to call.

Conclusion

There are other bottlenecks as well (I think messages are downloaded again after they have been sent), but I think the ones I listed are the one we should start from to improve performances.

I suggest there are two major things to do:

• Create functions to understand what we already have in the database (like takeTopMessageId)
• Use that functions to set offset for APIs

If I sound a bit rude I'm sorry, I don't want to, my English is a bit rustic. I really appreciate your work, I just want to try to help in improving it :)

What do you think?

Regards, R.

[realbardia]

Hi

Thank you for your great article. I read all of these. These are great ideas. We're currently working on the TelegramQML 2.0. It's different from the TelegramQML 1.0. We fix some of the above problems and some other not. But I'll add these changes and I'll include these change in the new TelegramQML.

Thank you very much