Open julian-klode opened 3 days ago
I think the real fix is to not lazily import modules like this in the standard library, but I assume there's a reason email.generators is lazily imported?
Since yesterday I have the same error on my servers, using Python 3.8. It happens to crash my unicorn workers resulting in bad gateway errors for end users. Until I manually restart my servers. Happened yesterday and also today in the morning.
It happened to one of our servers as well. Running:
We upgraded recently to Django 4.2.16, commit https://github.com/django/django/commit/bf4888d317ba4506d091eeac6e8b4f1fcc731199 might be related.
We run on Django 3.2.9 (..i know), got the following error log. Still no solution found for the problem.
[2024-09-18 03:41:11 +0200] [17792] [INFO] Booting worker with pid: 17792
[2024-09-18 04:31:45 +0200] [18105] [INFO] Booting worker with pid: 18105
[2024-09-18 06:27:00 +0200] [20686] [INFO] Booting worker with pid: 20686
[2024-09-18 06:27:01 +0200] [20686] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
self.load_wsgi()
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
self.wsgi = self.app.wsgi()
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
return self.load_wsgiapp()
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
return util.import_app(self.app_uri)
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
mod = importlib.import_module(module)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 848, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/var/www/crftwrkbn/crftr/crftr/wsgi.py", line 13, in <module>
from django.core.wsgi import get_wsgi_application
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/core/wsgi.py", line 2, in <module>
from django.core.handlers.wsgi import WSGIHandler
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 5, in <module>
from django.core.handlers import base
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 12, in <module>
from django.utils.log import log_response
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/utils/log.py", line 6, in <module>
from django.core import mail
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/core/mail/__init__.py", line 9, in <module>
from django.core.mail.message import (
File "/var/www/crftwrkbn/.venv/lib/python3.8/site-packages/django/core/mail/message.py", line 2, in <module>
from email import (
File "/usr/lib/python3.8/email/generator.py", line 17, in <module>
from email.errors import HeaderWriteError
ImportError: cannot import name 'HeaderWriteError' from 'email.errors' (/usr/lib/python3.8/email/errors.py)
[2024-09-18 06:27:01 +0200] [20686] [INFO] Worker exiting (pid: 20686)
[2024-09-18 06:27:04 +0200] [542] [INFO] Shutting down: Master
[2024-09-18 06:27:04 +0200] [542] [INFO] Reason: Worker failed to boot.
It sounds like those of you who were getting this error were upgrading Python on the file system without restarting your processes. If so, you should be able to get around the issue by restarting all Python processes to have the new code.
Thanks for your reply. Hmm this could mean that the server restart solved the problem. In our setup with 4 web servers it could hypothetically be that yesterday server-1 and server-2 were automatically updated, and server-3 and server-4 today so that it should not happen anymore after today. Let's wait until tomorrow morning to see if this assumption is correct.
Bug report
Bug description:
Pull request #122233 introduced a new class
HeaderWriteError
in commit 097633981879b3c9de9a1dd120d3aa585ecc2384 and imports that fromemail.generator
.This breaks running applications that have imported other parts of
email
before the update, and then try to import the generator past the update.Now this is a bit silly, but it is what
email.message.Message.as_string()
does, it importsemail.generator
inside the function - which may happen at any point of the program run-time rather than at startup.For example, the following pseudo-code will fail, assuming it has not generated another email earlier or manually imported the
email.generator
module.A particular instance of the issue is the
unattended-upgrades
package in Ubuntu and Debian, which will install the security update and then may send an email and fail there due to the ImportError, see https://bugs.launchpad.net/ubuntu/+source/python3.8/+bug/2080940.I'm wondering if it's feasible to add a workaround to the stable branches:
Cchange the email.generator module import:
to graciously support the previous version email.errors:
This is a safe change, existing applications, where the import fails can't be having
except HeaderWriteError
statements anyway.Thanks.
CPython versions tested on:
3.12
Operating systems tested on:
No response