Cannot send Challenge request emails - Python version pinned until Django 4.2.12 is released

[jmsmkn]

Emails with special characters are failing to get sent.

To reproduce make a challenge request, the email tries to include a euro symbol (€) in the line. We also see this when trying to send email to some users with non-ascii names.

{"exc_type": "UnicodeEncodeError", "exc_message": ["utf-8", "- Base cost: 5000 \udce2\udc82\udcac", 18, 21, "surrogates not allowed"], "exc_module": "builtins"}

Traceback (most recent call last):
File "/opt/poetry/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
R = retval = fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/sentry_sdk/integrations/celery.py", line 312, in _inner
reraise(*exc_info)
File "/opt/poetry/.venv/lib/python3.11/site-packages/sentry_sdk/_compat.py", line 127, in reraise
raise value
File "/opt/poetry/.venv/lib/python3.11/site-packages/sentry_sdk/integrations/celery.py", line 307, in _inner
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
return self.run(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/djcelery_email/tasks.py", line 57, in send_emails
send_emails.retry([[message], combined_kwargs], exc=e, throw=False)
File "/opt/poetry/.venv/lib/python3.11/site-packages/celery/app/task.py", line 736, in retry
raise_with_context(exc)
File "/opt/poetry/.venv/lib/python3.11/site-packages/djcelery_email/tasks.py", line 48, in send_emails
sent = conn.send_messages([dict_to_email(message)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/django_ses/__init__.py", line 163, in send_messages
kwargs = self._get_send_email_parameters(message, source, email_feedback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/django_ses/__init__.py", line 245, in _get_send_email_parameters
else self._get_v1_parameters(message, source))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/django_ses/__init__.py", line 282, in _get_v1_parameters
RawMessage={'Data': dkim_sign(message.message().as_string(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/django/core/mail/message.py", line 133, in as_string
g.flatten(self, unixfrom=unixfrom, linesep=linesep)
File "/usr/local/lib/python3.11/email/generator.py", line 115, in flatten
self._write(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 180, in _write
self._dispatch(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 217, in _dispatch
meth(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 275, in _handle_multipart
g.flatten(part, unixfrom=False, linesep=self._NL)
File "/usr/local/lib/python3.11/email/generator.py", line 115, in flatten
self._write(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 180, in _write
self._dispatch(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 217, in _dispatch
meth(msg)
File "/usr/local/lib/python3.11/email/generator.py", line 246, in _handle_text
msg.set_payload(msg._payload, charset)
File "/opt/poetry/.venv/lib/python3.11/site-packages/django/core/mail/message.py", line 168, in set_payload
has_long_lines = any(
^^^^
File "/opt/poetry/.venv/lib/python3.11/site-packages/django/core/mail/message.py", line 169, in <genexpr>
len(line.encode()) > RFC5322_EMAIL_LINE_LENGTH_LIMIT
^^^^^^^^^^^^^
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 18-20: surrogates not allowed

[jmsmkn]

This appears to be https://code.djangoproject.com/ticket/35361

[jmsmkn]

A fix for this will be available in https://docs.djangoproject.com/en/5.0/releases/4.2.12/ (Expected May 5th). Until then, we will need to pin our python version.

[jmsmkn]

Confirmed that the workaround is working.