Closed gfd2020 closed 10 months ago
Thank you @gfd2020, this would be very useful!
But, there is an existing open ticket for this (#693) and I think @thalespr was working on it, not sure about his progress on it. Can I close this as duplicate and could you work together with him?
Thank you @gfd2020, this would be very useful!
But, there is an existing open ticket for this (#693) and I think @thalespr was working on it, not sure about his progress on it. Can I close this as duplicate and could you work together with him?
Oh! I didn't know that already had a ticket like that, sorry. You can close then. My implementation is already well advanced. Only the media cases remain to be dealt with, text is already working. I can work with him.
Ok, thank you, let's continue the conversation on #693.
Processing a large WhatsApp Android DB (provided by @aberenguel), it is taking ~4 hours to extract quoted messages, while the "regular" messages are extracted in ~50 seconds. The database file has ~1.8 GB. It contains ~1.12M messages, 61K chats and 168K quoted messages.
It is a similar situation described in #1889. With a huge number of chats (61K), the query to get quoted messages will be executed 61K times. This query is taking between 200 and 500 ms, which is relatively fast, but multiplied by 61K, it will take several hours. This is an extreme case, but having a few hundred chats is somewhat common.
I applied the same technique used in #1889, querying just once, not per chat. It reduced the time to extract quoted messages to only 10 seconds. I am including this in PR #2048.
Processing a large WhatsApp Android DB (provided by @aberenguel), it is taking ~4 hours to extract quoted messages, while the "regular" messages are extracted in ~50 seconds. The database file has ~1.8 GB. It contains ~1.12M messages, 61K chats and 168K quoted messages.
It is a similar situation described in #1889. With a huge number of chats (61K), the query to get quoted messages will be executed 61K times. This query is taking between 200 and 500 ms, which is relatively fast, but multiplied by 61K, it will take several hours. This is an extreme case, but having a few hundred chats is somewhat common.
I applied the same technique used in #1889, querying just once, not per chat. It reduced the time to extract quoted messages to only 10 seconds. I am including this in PR #2048.
Thank you very much @wladimirleite for investigating, debugging times and fixing this performance regression!
Whatsapp allows you to quote messages. It would be interesting if IPED could detect these messages to improve the context of conversations.
I'm already doing a PR to address this. Example below: The behavior is the same as on WhatsApp. When clicking, the conversation is scrolled and marked the message with a color that disappears.
https://github.com/sepinf-inc/IPED/assets/59742865/9f9e5d4f-d231-4014-846b-db22c57a5a1a