How can I know if there are still pending emails

[sergani]

Hi there, I'm using the Dynamic Manager; 5 workers, 10 divisor. I'm able to check if the mailer is running or not using Mailer.running. However, I would like to know if there's a way to check if there are actual SMTP connections in place, or pending emails to be sent, before attempting to shutdown Mailer, using Mailer.stop(). The other reason is because i don't want overlapping instances running at the same time which may get my system blocked by the mail provider.

Appreciate your thoughts.

Thanks //M

[amcgregor]

If you can retrieve the DynamicManager instance associated with your Mailer instance via mailer.manager, the _work_queue is accessible, as is the collection of _threads. Just be very careful, if manipulating these collections in any way, to utilize the _management_lock as appropriate and demonstrated further down when adding threads. Note that the size (number of pending deliveries) will be accessible via the queue's qsize() method.

The number of threads within the set indicates the number of active deliveries plus sleeping threads, waiting for work, that have not starved to death, yet. AFIK there is no way without external tracking yourself, e.g. via logging capture, to more accurately track "in-flight status", excepting "delivery complete", achievable by adding a "done callback" to the receipt you are given when you enqueue the message for delivery.

mailer = Mailer(...)
message = mailer.new(...)
receipt = message.send()  # or mailer.send(message)

def capture_result(receipt):
    print(receipt.result())

receipt.add_done_callback(capture_result)

In this way you can maintain a counter externally for very accurate tracking. Increment and enqueue, in a done callback, decrement. Locking should not be necessary for "atomic" += 1 and -= 1 updates, if my interpretation of GIL operation is correct. 😉

Important: This is also how you implement error capture. Reference the Futures documentation; the receipt is a Future instance.

The SMTP transport instance would be accessible via the manager's own transport attribute (which is actually a TransportPool instance), and lo, from there, transports is the attribute containing the Queue (ref: qsize()) of active or available connections within the pool.

On the overlapping instances thing, some frameworks offer registrable "utility singletons" (e.g. Pyramid) to help de-duplicate, my own applications use a global Mailer instance shared via a module dedicated to such singletons. With any form of scaling pool executor, you are making multiple connections in parallel. "Overlapping instances" — this is the entire point of these background execution delivery managers. 😕

Does this help answer your questions?

[amcgregor]

Closing issue due to inactivity; candidate for Wiki documentation.