search

KumoCorp / kumomta

The first Open-Source high-performance MTA developed from the ground-up for high-volume email sending environments.
https://kumomta.com
Apache License 2.0
202 stars 25 forks source link

Configurable queue sleep duration #232

Open edgarsendernet opened 4 days ago

edgarsendernet commented 4 days ago

https://github.com/KumoCorp/kumomta/blob/91ce59628d61fd176c6f19c75d1d869066cb10e3/crates/kumod/src/queue.rs#L1326

Instead of a hardcoded value of 60s, this could be exposed as a configuration variable. A user might want to reduce sleep delay to reduce message latency at the expense of slighly higher CPU usage.

wez commented 3 days ago

What are you really trying to optimize here? What's the concrete use case?

Note that when a message lands in the scheduled queue, it is typically because the message cannot be immediately delivered due to the disposition of the destination site. Having a 1 minute upper bound on promoting those delayed messages to the ready queue is pretty reasonable.

The code you pointed to above is for the case where the scheduled queue is empty and we're waiting for more messages to land in the scheduled queue, and they will typically have a next due time of much greater than 1 minute from the time they are inserted into the scheduled queue, so it feels weird to want to try to optimize that case.

edgarsendernet commented 3 days ago

We were running some tests to measure end to end latency, especially under load. There are other instances of the same hardcoded 60s sleep delay (for example, if the result is PopResult::Sleep too), so had a suspicion that this might be the cause.