Support for async init of handler

[kesavkolla]

Currently there are only two mechanisms to instantiate handler either from_cli or init. Both of these methods are regular methods they don't support async. Can you please add support for a async initialization of handler so that we can do some initialization tasks which needs await.

Currently I've a requirement of creating redis connection in handler. I can't use aioredis create_pool inside my handler init as there is no way to do await.

[pepoluan]

I think I have some ideas on how to implement this, but that'll require some deep changes in the underlying mechanism.

[pepoluan]

@kesavkolla I'm not familiar with aioredis, so I have to ask: Can you call create_pool multiple times? Will there be a deleterious effect?

Because if you can do that without major negative impact, you can put the call to create_pool inside handle_HELO / handle_EHLO, which are already required to be async.

Please CMIIW because, as I said, I am not really familiar with aioredis.

[waynew]

I'm pretty sure that you could do something like:

asyncio.run(aioredis.create_pool)
controller.start()

Is there any reason that won't work for you?

[pepoluan]

Won't asyncio.run block until aioredis.create_pool completes?

If that is the case, then probably kesavkolla wants the initialization of aioredis pool to happen concurrently with the initialization of aiosmtpd.

(Please CMIIW, I'm a bit unclear about asyncio.run's behavior.)

[waynew]

Ah, it might be create_task. I know you can schedule a task, either with a high level or low level call. It may take some experiments to figure out what calls

[pepoluan]

A 'simple' workaround would be to create a hook, let's say handle__init__ (notice the pair of dunders), which can be await-ed from SMTP.__init__

The handler class must then implement a semaphore-like system to ensure that other hooks won't start processing until handle__init__ completes. Something like

async def handle_AUTH(...):
    while not self.redis_ready:
        await asyncio.sleep(0.1)
    ... rest of code ...

after all, we can't babysit the handler classes... they are "consenting adults" anyways 😄

[pepoluan]

That said...

Why not just create an event loop, have the loop do call_soon to the handler's initialization (which involves aioredis.create_pool), then feed the loop to Controller?

If the semaphore-like system as I posted above has been implemented, then the handler hooks will politely wait until aioredis.create_pool is done ...

The more we talk about this, the more I think we don't really need to change SMTP at all...

[astapelfeld]

Wouldn't all the proposed solutions create a pool every time a client connects to the server? Is there any way to use one pool that is created when the server is started? In my case, I want to connect to mysql to check recipients in the RCPT handler and extract data from the body and store it in mysql in the DATA handler.