Sending emails via IONOS fails

[Cameo007]

Description

I am using a mail account at IONOS for my Matrix instance. It worked in the past but for some time I see an error message in Synapse that the email cannot be sent.

When I open a ticket I receive the information from IONOS that the email was rejected by our SMTP server because it does not comply with the defined standards from RFC5321 and RFC5322.

Cause:
Your email was rejected by our SMTP server because it does not comply with the defined standards from RFC5321 and RFC5322.

Solution:
Often, this error occurs when you use an address book entry or autocomplete in your email program for the recipient's email address.

Therefore, you should test the issue again by entering the recipient's email address manually this time. If it does not fail anymore, either the address book entry must be deleted and recreated or the entry must be removed from the auto-completion for the permanent error removal.

Technical Explanation:
The email client must meet the following criteria regarding sending emails:

The headers of your email must contain a valid "Date" header according to [RFC2822 section 3.3](https://datatracker.ietf.org/doc/html/rfc2822#section-3.3).
The "Date" header must only be there once
The following headers must be present at most only once: "From", "Sender", "To", "CC", "Subject"
If the above headers are used, they must not be empty and must be syntactically correct.
Please note: The headers "To" and "CC" can of course contain multiple recipients.

We adhere to the recommendations from RFC7103 and do not perform automatic correction of erroneous messages because this invalidates DKIM signatures.

Steps to reproduce

Use IONOS to send emails

Homeserver

mintux.de

Synapse Version

1.102.0 (also before)

Installation Method

Docker (matrixdotorg/synapse)

Database

single PostgreSQL (not restored or ported)

Workers

I don't know

Platform

NixOS 23.11 via Docker on a DELL Laptop

Configuration

No response

Relevant log output

2024-03-06 21:36:26,337 - synapse.handlers.send_email - 215 - INFO - POST-54 - Sending email to xx@gmx.net
2024-03-06 21:36:26,871 - synapse.handlers.identity - 398 - ERROR - POST-54 - Error sending threepid validation email to xx@gmx.net
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/identity.py", line 396, in send_threepid_validation
    await send_email_func(email_address, token, client_secret, session_id)
  File "/usr/local/lib/python3.11/site-packages/synapse/push/mailer.py", line 236, in send_add_threepid_mail
    await self.send_email(
  File "/usr/local/lib/python3.11/site-packages/synapse/push/mailer.py", line 363, in send_email
    await self.send_email_handler.send_email(
  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/send_email.py", line 217, in send_email
    await self._sendmail(
  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/send_email.py", line 131, in _sendmail
    await make_deferred_yieldable(d)
twisted.mail._except.SMTPDeliveryError: 554 Transaction failed
Reject due to policy restrictions.
For explanation visit https://www.ionos.com/help/index.php?id=2425
>>> .
<<< 554-Transaction failed
<<< 554-Reject due to policy restrictions.
<<< 554 For explanation visit https://www.ionos.com/help/index.php?id=2425

Anything else that would be useful to know?

No response

[reivilibre]

What does your e-mail sending config look like? Feel free to redact the values that are sensitive but at least show what's there.

A quick poke shows we do send a Date header and we don't put From, To or Subject more than once.

[Cameo007]

This is my email config.

[reivilibre]

I wonder if this is because, unless I'm mistaken, Synapse sends e-mails with a LF line ending whereas the standard says it should use CRLF.

Are you happy to try applying and running a patch?

In synapse/handlers/send_email.py , at the very bottom,

        await self._sendmail(
            self._reactor,
            self._smtp_host,
            self._smtp_port,
            raw_from,
            raw_to,
            multipart_msg.as_string().encode("utf8"),
            username=self._smtp_user,
            password=self._smtp_pass,
            require_auth=self._smtp_user is not None,
            require_tls=self._require_transport_security,
            enable_tls=self._enable_tls,
            force_tls=self._force_tls,
        )

could be replaced with

        await self._sendmail(
            self._reactor,
            self._smtp_host,
            self._smtp_port,
            raw_from,
            raw_to,
            multipart_msg.as_string().encode("utf8").replace(b"\r", b"").replace(b"\n", b"\r\n"),
            username=self._smtp_user,
            password=self._smtp_pass,
            require_auth=self._smtp_user is not None,
            require_tls=self._require_transport_security,
            enable_tls=self._enable_tls,
            force_tls=self._force_tls,
        )

Though to be honest, I don't know if I'm correct or not; likely I'd need to spend more time with this.

[Cameo007]

I'm right now building a custom docker image with this change. We will see if it works or not.

[Cameo007]

No, it doesn't help.

[reivilibre]

Thanks for trying that, sorry it didn't help.

I'm not really sure what to suggest here; I don't have any direct suggestions with that error.

You could try to intercept the connection (tcpdump or Wireshark) and exactly what it's sending, byte-for-byte? I guess you will need something like MITM proxy since the SMTP traffic is encrypted.

There is probably going to be a fair amount of staring at it and maybe comparing it against another application that does manage to send e-mail through this SMTP host.