search

matrix-org / sydent

Sydent: Reference Matrix Identity Server
http://matrix.org
Apache License 2.0
303 stars 84 forks source link

Sentry error involving email address with non-ascii characters #458

Open DMRobertson opened 3 years ago

DMRobertson commented 3 years ago

Spotted by sentry: https://sentry.matrix.org/sentry/sydent/issues/234969/?query=release%3A%22sydent%402.4.6%20(b%3DHEAD%2C1c0ae92)%22

The email address in question contains the character è encoded as UTF-8 bytes C3 A8. We end up calling smtp.sendmail which tries to encode this in ASCII and fails.

I'm not familiar with the specific requirements of email addresses. There's a comprehensive-looking Stack Overflow answer which suggests it's possible with an option called SMTPUTF8.

Oh, and while we're at it, we can use logger.exception to get better sentry reports.

traceback:

Nov  4 11:17:39 corus sydent-vectoris[29785]: 2021-11-04 11:17:39,011 - sydent.util.emailutils - 105 - INFO - Sending mail to  EMAILADDRESS with mail server: relay.matrix.org
Nov  4 11:17:39 corus sydent-vectoris[29785]: Unhandled Error
Nov  4 11:17:39 corus sydent-vectoris[29785]: Traceback (most recent call last):
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/env/lib/python3.7/site-packages/twisted/web/server.py", line 294, in render
Nov  4 11:17:39 corus sydent-vectoris[29785]:     body = resrc.render(self)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/env/lib/python3.7/site-packages/twisted/web/resource.py", line 263, in render
Nov  4 11:17:39 corus sydent-vectoris[29785]:     return m(request)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/__init__.py", line 152, in inner
Nov  4 11:17:39 corus sydent-vectoris[29785]:     return dict_to_json_bytes(f(self, request))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/store_invite_servlet.py", line 169, in render_POST
Nov  4 11:17:39 corus sydent-vectoris[29785]:     sendEmail(self.sydent, templateFile, normalised_address, substitutions)
Nov  4 11:17:39 corus sydent-vectoris[29785]: --- <exception caught here> ---
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/util/emailutils.py", line 124, in sendEmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     smtp.sendmail(mailFrom, mailTo, mailString.encode("utf-8"))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 872, in sendmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     (code, resp) = self.rcpt(each, rcpt_options)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 542, in rcpt
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.putcmd("rcpt", "TO:%s%s" % (quoteaddr(recip), optionlist))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 367, in putcmd
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.send(str)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 352, in send
Nov  4 11:17:39 corus sydent-vectoris[29785]:     s = s.encode(self.command_encoding)
Nov  4 11:17:39 corus sydent-vectoris[29785]: builtins.UnicodeEncodeError: 'ascii' codec can't encode character '\xe8' in position 12: ordinal not in range(128)
Nov  4 11:17:39 corus sydent-vectoris[29785]: 2021-11-04 11:17:39,056 - twisted - 147 - CRITICAL - Unhandled Error
Nov  4 11:17:39 corus sydent-vectoris[29785]: Traceback (most recent call last):
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/env/lib/python3.7/site-packages/twisted/web/server.py", line 294, in render
Nov  4 11:17:39 corus sydent-vectoris[29785]:     body = resrc.render(self)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/env/lib/python3.7/site-packages/twisted/web/resource.py", line 263, in render
Nov  4 11:17:39 corus sydent-vectoris[29785]:     return m(request)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/__init__.py", line 152, in inner
Nov  4 11:17:39 corus sydent-vectoris[29785]:     return dict_to_json_bytes(f(self, request))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/store_invite_servlet.py", line 169, in render_POST
Nov  4 11:17:39 corus sydent-vectoris[29785]:     sendEmail(self.sydent, templateFile, normalised_address, substitutions)
Nov  4 11:17:39 corus sydent-vectoris[29785]: --- <exception caught here> ---
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/util/emailutils.py", line 124, in sendEmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     smtp.sendmail(mailFrom, mailTo, mailString.encode("utf-8"))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 872, in sendmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     (code, resp) = self.rcpt(each, rcpt_options)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 542, in rcpt
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.putcmd("rcpt", "TO:%s%s" % (quoteaddr(recip), optionlist))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 367, in putcmd
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.send(str)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 352, in send
Nov  4 11:17:39 corus sydent-vectoris[29785]:     s = s.encode(self.command_encoding)
Nov  4 11:17:39 corus sydent-vectoris[29785]: builtins.UnicodeEncodeError: 'ascii' codec can't encode character '\xe8' in position 12: ordinal not in range(128)
Nov  4 11:17:39 corus sydent-vectoris[29785]: 2021-11-04 11:17:39,070 - sydent.http.servlets - 157 - ERROR - Exception processing request
Nov  4 11:17:39 corus sydent-vectoris[29785]: Traceback (most recent call last):
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/util/emailutils.py", line 124, in sendEmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     smtp.sendmail(mailFrom, mailTo, mailString.encode("utf-8"))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 872, in sendmail
Nov  4 11:17:39 corus sydent-vectoris[29785]:     (code, resp) = self.rcpt(each, rcpt_options)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 542, in rcpt
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.putcmd("rcpt", "TO:%s%s" % (quoteaddr(recip), optionlist))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 367, in putcmd
Nov  4 11:17:39 corus sydent-vectoris[29785]:     self.send(str)
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/usr/local/lib/python3.7/smtplib.py", line 352, in send
Nov  4 11:17:39 corus sydent-vectoris[29785]:     s = s.encode(self.command_encoding)
Nov  4 11:17:39 corus sydent-vectoris[29785]: UnicodeEncodeError: 'ascii' codec can't encode character '\xe8' in position 12: ordinal not in range(128)
Nov  4 11:17:39 corus sydent-vectoris[29785]: The above exception was the direct cause of the following exception:
Nov  4 11:17:39 corus sydent-vectoris[29785]: Traceback (most recent call last):
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/__init__.py", line 152, in inner
Nov  4 11:17:39 corus sydent-vectoris[29785]:     return dict_to_json_bytes(f(self, request))
Nov  4 11:17:39 corus sydent-vectoris[29785]:   File "/opt/vectoris/sydent/sydent/http/servlets/store_invite_servlet.py", line 169, in render_POST
Nov  4 11:17:39 corus sydent-vectoris[29785]:     sendEmail(self.sydent, templateFile, normalised_address, substitutions)