Aiosmtpd incompatible with O365 service

[cnicodeme]

Hi,

When receiving the command "MAIL FROM", the code located at smtp.py is checking for parameters such as SIZE, SMTPUTF8, etc.

The problem is if it receives an unknown paramter, the code fails at L1278:

        if len(params) > 0:
            await self.push(
                '555 MAIL FROM parameters not recognized or not implemented')
            return

By using this behavior, which is repeated for the RCPT command (L1319), there is absolutely no way to accept an email that contains parameters that AIOSmtpd handles, even valid parameters.

The issue is that service, like O365, seems to send the MAIL FROM command with parameters that are unknown to AIOSmtpd, causing the command to be refused and the email never delivered.

I believe that the parameters sent by O365 are valid and should be either accepted, or ignored, but not refused.

[waynew]

I was thrilled when I saw this issue because I've had issues with sending email from work to my aiostmpd server....

However, I'm not getting the log messages that I thought I would see. Do you have examples of other parameters that o365 tries to send?

[cnicodeme]

YES! Following my ticket, I've added tracking to the remaining parameters to know which one was causing issues.

It's AUTH=<>

Don't ask me why though :p

But in a broader sense, I don't think it's a good idea to block the command if there are any unknown parameters. I can understand why it would be important to be able to process them all (I mean, they are here for a reason ... well, except O365), but AIOSmtpd does not support all the known parameters so it can easily break with a system sending one of the unsupported parameter.

What I think would be better, is AIOSmtpd calling a hook that handles the parameters once AIOSmptd has finished treating the one it knows. That hook could either send the whole set of parameters, or the remaining one, and then, only if that hook returns a non empty object (the untreated one), then AIOSmtpd could eventually break and return a 550.

If I've understood the RFC correctly, there are a list of official mail-parameters that can be accepted, and they are listed here: https://www.rfc-editor.org/rfc/rfc6729.html#section-6.2

(again, if I understood correctly, not entirely sure), but again, it would be best to let the developer using AIOSmptd decide what to do in the event an unknown parameter is received.

[waynew]

I'm OK with us offering a hook and falling back to failure/rejection.

Interestingly enough I was having problems with getting email from outlook on my own server, but that was because I wasn't properly returning \r\n, and apparently Outlook is strict about those bytes :upside_down_face:

[cnicodeme]

I think it's the best solution. I recently had another notification about extra headers :

NOTIFY="SUCCESS, FAILURE"

(I don't know which server these came from though, it's possible it's not O365, but still, in the current implementation of Aiosmtpd, the email would have been refused).

As for O365, we did had a few issues in the past too. If you rewrite the email using the Python library, some might break, they are strict on the headers, even the "Received" one. But even though they are difficult to work with, I kind of like it : they enforce a proper structure to respect and force developers to adhere to the standard more. Truth is, if your email works at Outlook, it should work everywhere else too.

[leandroltavares]

Hey guys, any updates on this? I wonder if the addition of the hook is planned to be added and released?

[nim-odoo]

If I understand correctly https://www.rfc-editor.org/rfc/rfc4954#section-5

For this reason, servers that advertise support for this extension MUST support the AUTH parameter to the MAIL FROM command even when the client has not authenticated itself to the server.

Since aiosmtpd advertises AUTH LOGIN PLAIN by default, it should not reject mails using AUTH in their MAIL FROM.

[nim-odoo]

Still early, but it seems the following fixes the issue related to AUTH=<>:

    async def handle_EHLO(self, server, session, envelope, hostname, response):
        session.host_name = hostname
        return [r for r in response if not r.startswith("250-AUTH")]

But of course it only works if clients properly implement the RFC.

It applies if using aiostmpd, though. When using it as a library, one can override _auth_methods in SMTP.