paperless-ngx 1.17.0 CERTIFICATE_VERIFY_FAILED

[f0x52]

Describe the bug

After updating paperless-ngx to 1.17.0 (from 1.16.5), the mail fetching tasks stopped working, with the following ssl related error in the logs:

Aug 20 19:20:53 cosmos gunicorn[2859903]: [2023-08-20 19:20:52,997] [ERROR] [django.request] Internal Server Error: /api/mail_accounts/test/
Aug 20 19:20:53 cosmos gunicorn[2859903]: Traceback (most recent call last):
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/zpw6zvxy6kcm3r6rx54pa2irjjjy3nfk-python3.10-asgiref-3.7.2/lib/python3.10/site-packages/asgiref/sync.py", line 534, in thread_handler
Aug 20 19:20:53 cosmos gunicorn[2859903]:     raise exc_info[1]
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/g9khwc4vzvxczchnb8r6qgfrhzn1k1hh-python3.10-Django-4.2.4/lib/python3.10/site-packages/django/core/handlers/exception.py", line 42, in inner
Aug 20 19:20:53 cosmos gunicorn[2859903]:     response = await get_response(request)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/g9khwc4vzvxczchnb8r6qgfrhzn1k1hh-python3.10-Django-4.2.4/lib/python3.10/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
Aug 20 19:20:53 cosmos gunicorn[2859903]:     response = await wrapped_callback(
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/zpw6zvxy6kcm3r6rx54pa2irjjjy3nfk-python3.10-asgiref-3.7.2/lib/python3.10/site-packages/asgiref/sync.py", line 479, in __call__
Aug 20 19:20:53 cosmos gunicorn[2859903]:     ret: _R = await loop.run_in_executor(
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/zpw6zvxy6kcm3r6rx54pa2irjjjy3nfk-python3.10-asgiref-3.7.2/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 40, in run
Aug 20 19:20:53 cosmos gunicorn[2859903]:     result = self.fn(*self.args, **self.kwargs)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/zpw6zvxy6kcm3r6rx54pa2irjjjy3nfk-python3.10-asgiref-3.7.2/lib/python3.10/site-packages/asgiref/sync.py", line 538, in thread_handler
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return func(*args, **kwargs)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/g9khwc4vzvxczchnb8r6qgfrhzn1k1hh-python3.10-Django-4.2.4/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return view_func(*args, **kwargs)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/g9khwc4vzvxczchnb8r6qgfrhzn1k1hh-python3.10-Django-4.2.4/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return self.dispatch(request, *args, **kwargs)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p1g03gnbhkqxgjiw8dm5fv4v48vmjj7h-python3.10-djangorestframework-3.14.0/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
Aug 20 19:20:53 cosmos gunicorn[2859903]:     response = self.handle_exception(exc)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p1g03gnbhkqxgjiw8dm5fv4v48vmjj7h-python3.10-djangorestframework-3.14.0/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self.raise_uncaught_exception(exc)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p1g03gnbhkqxgjiw8dm5fv4v48vmjj7h-python3.10-djangorestframework-3.14.0/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
Aug 20 19:20:53 cosmos gunicorn[2859903]:     raise exc
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p1g03gnbhkqxgjiw8dm5fv4v48vmjj7h-python3.10-djangorestframework-3.14.0/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
Aug 20 19:20:53 cosmos gunicorn[2859903]:     response = handler(request, *args, **kwargs)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/kz2iq03l3ykwj0lxy1cgzfzl38phslqv-paperless-ngx-1.17.0/lib/paperless-ngx/src/paperless_mail/views.py", line 64, in post
Aug 20 19:20:53 cosmos gunicorn[2859903]:     with get_mailbox(
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/kz2iq03l3ykwj0lxy1cgzfzl38phslqv-paperless-ngx-1.17.0/lib/paperless-ngx/src/paperless_mail/mail.py", line 403, in get_mailbox
Aug 20 19:20:53 cosmos gunicorn[2859903]:     mailbox = MailBox(server, port, ssl_context=ssl.create_default_context())
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p510fiprs4qqiw34007zknc6s6sf3jir-python3.10-imap-tools-1.2.0/lib/python3.10/site-packages/imap_tools/mailbox.py", line 296, in __init__
Aug 20 19:20:53 cosmos gunicorn[2859903]:     super().__init__()
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p510fiprs4qqiw34007zknc6s6sf3jir-python3.10-imap-tools-1.2.0/lib/python3.10/site-packages/imap_tools/mailbox.py", line 32, in __init__
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self.client = self._get_mailbox_client()
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/p510fiprs4qqiw34007zknc6s6sf3jir-python3.10-imap-tools-1.2.0/lib/python3.10/site-packages/imap_tools/mailbox.py", line 302, in _get_mailbox_client
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return imaplib.IMAP4_SSL(self._host, self._port, self._keyfile, self._certfile, self._ssl_context,
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/imaplib.py", line 1323, in __init__
Aug 20 19:20:53 cosmos gunicorn[2859903]:     IMAP4.__init__(self, host, port, timeout)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/imaplib.py", line 202, in __init__
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self.open(host, port, timeout)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/imaplib.py", line 1336, in open
Aug 20 19:20:53 cosmos gunicorn[2859903]:     IMAP4.open(self, host, port, timeout)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/imaplib.py", line 312, in open
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self.sock = self._create_socket(timeout)
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/imaplib.py", line 1327, in _create_socket
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return self.ssl_context.wrap_socket(sock,
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/ssl.py", line 513, in wrap_socket
Aug 20 19:20:53 cosmos gunicorn[2859903]:     return self.sslsocket_class._create(
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/ssl.py", line 1071, in _create
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self.do_handshake()
Aug 20 19:20:53 cosmos gunicorn[2859903]:   File "/nix/store/2c7sgx69p6mmp76cvmi5j6c72dj76jj8-python3-3.10.12/lib/python3.10/ssl.py", line 1342, in do_handshake
Aug 20 19:20:53 cosmos gunicorn[2859903]:     self._sslobj.do_handshake()
Aug 20 19:20:53 cosmos gunicorn[2859903]: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

Downgrading to 1.16.5 resulted in email working again.

Notify maintainers

@lukegb @gador @erikarvstedt

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[cosmos] root [~] nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.42, NixOS, 23.05 (Stoat), 23.05.2918.4cdad15f34e6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.5`
 - channels(f0x): `"nixos-unstable"`
 - channels(root): `"nixos-23.05"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[leona-ya]

Sorry for the delay, haven't seen this Issue (am only in unstable maintainers for paperless-ngx, not 23.05 yet). Will take a look at it, and I think I have an idea on how to fix this