linagora / tmail-backend

GNU Affero General Public License v3.0
36 stars 17 forks source link

[Experiment] INBOX archival task improvement #859

Open quantranhong1999 opened 9 months ago

quantranhong1999 commented 9 months ago

Why

Today's behavior: iterating all user INBOX messages using Cassandra which put much pressure on Cassandra.

Using OpenSearch to query the date could avoid iterating all the messages which could bring faster response time (in most of the cases?).

Following Benoit's concern: OpenSearch may not be good for searching a big INBOX which searchs through a lot of shards, and OpenSearch is not a source of truth.

My proposal:

I feel that solution 1 could be a safer solution while still bringing improvement in task speed.

DoD

Arsnael commented 9 months ago

Hmm... I'm wondering if it's really necessary to go that far. I would think the moment you have potentially a lot of pressure is the first time you run the task. After when a good part of your messages are archived already, the pressure would not be so much?

quantranhong1999 commented 9 months ago

Hmm... I'm wondering if it's really necessary to go that far. I would think the moment you have potentially a lot of pressure is the first time you run the task. After when a good part of your messages are archived already, the pressure would not be so much?

I agree. But still a potential improvement IMO, I record the idea otherwise one day I forget it. Open for discussion ^^. Not priority though for sure.