To save allocating/deallocating RAM, The Envelopes get recycled for each email transaction. Each envelope is borrowed from a pool, and then returned once the middleware is finished saving the email(s). The problem was noticed when a server suffered hard disk degradation, causing each envelope to be processed for more than 30 seconds. This in turn triggered a gateway timeout (set by the gw_save_timeout config prop), which didn't wait for the Envelope to release the lock before continuing with a new transaction for the same connection. Because the Envelope was locked, this causes a race condition and quickly exhausted all envelopes, causing the server to freeze. Additionally, workerMsg were not being returned to the pool after a timeout
Todo
When a gateway timeout occurs, client should receive the timeout message immediately, but wait until the processing is finished and the resource is available, before continuing with another transaction.
After the processing finished, make sure to put workerMsg back in the pool.
To save allocating/deallocating RAM, The Envelopes get recycled for each email transaction. Each envelope is borrowed from a pool, and then returned once the middleware is finished saving the email(s). The problem was noticed when a server suffered hard disk degradation, causing each envelope to be processed for more than 30 seconds. This in turn triggered a gateway timeout (set by the
gw_save_timeoutconfig prop), which didn't wait for the Envelope to release the lock before continuing with a new transaction for the same connection. Because the Envelope was locked, this causes a race condition and quickly exhausted all envelopes, causing the server to freeze. Additionally,workerMsgwere not being returned to the pool after a timeoutTodo
workerMsgback in the pool.