search

OfflineIMAP / offlineimap3

Read/sync your IMAP mailboxes (python3)
Other
455 stars 64 forks source link

TLS/SSL connection has been closed (EOF) with TLSv1 #118

Open berghaus opened 2 years ago

berghaus commented 2 years ago

I think this is an upstream issue:

though please correct me if I am wrong :-)

General information

Configuration file offlineimaprc

[general]
accounts = CERN

[Account CERN]
localrepository = CERNlocal
remoterepository = CERNremote

[Repository CERNlocal]
type = Maildir
localfolders = ~/Mail/CERN

[Repository CERNremote]
type = IMAP
remotehost = imap.cern.ch
remoteport = 993
remoteuser = XXXXXXX
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
# SHA1 Fingerprint=46:6B:D1:77:49:B6:8D:F9:55:37:4B:7A:78:40:11:E1:A9:76:D6:57
# cert_fingerprint = 466bd17749b68df955374b7a784011e1a976d657
folderfilter = lambda folder: folder not in ['Calendar','Calendar/Birthdays','Calendar/Sub Folder 1','Calendar/Sub Folder 2','Calendar/United States holidays','Contacts','Contacts/Sub Folder 1','Contacts/Sub Folder 2','Contacts/Skype for Business Contacts','Deleted Items','Drafts','Journal','Junk Email','Notes','Outbox','Sync Issues','Sync Issues/Conflicts','Sync Issues/Local Failures','Sync Issues/Server Failures','Tasks','Tasks/Sub Folder 1','Tasks/Sub Folder 2']

Note: I have played with setting ssl_version and disabling starttls.

Logs, error

OfflineIMAP 7.3.4
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.101 (bundled), Python v3.10.4, OpenSSL 3.0.2 15 Mar 2022
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync CERN:
 [thread]: Register new thread 'Account sync CERN' (account 'CERN')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account CERN
 Establishing connection to imap.cern.ch:993 (CERNremote)
 [imap]: CERNremote: level 'tls_compat', version 'None'
 ERROR: While attempting to sync account 'CERN'
  TLS/SSL connection has been closed (EOF) (_ssl.c:997)
 ['  File "~/.local/lib/python3.10/site-packages/offlineimap/accounts.py", line 293, in syncrunner\n    self.__sync()\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/accounts.py", line 372, in __sync\n    remoterepos.getfolders()\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/imapserver.py", line 557, in acquireconnection\n    imapobj = imaplibutil.WrappedIMAP4_SSL(\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/imaplibutil.py", line 202, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 2191, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 360, in __init__\n    self.open(host, port)\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/imaplibutil.py", line 210, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 2204, in open\n    self.ssl_wrap_socket()\n', '  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 554, in ssl_wrap_socket\n    self.sock = ctx.wrap_socket(self.sock, server_hostname=self.host)\n', '  File "/usr/lib/python3.10/ssl.py", line 512, in wrap_socket\n    return self.sslsocket_class._create(\n', '  File "/usr/lib/python3.10/ssl.py", line 1070, in _create\n    self.do_handshake()\n', '  File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account 'CERN' in 0:00
[thread]: Unregister thread 'Account sync CERN'
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'CERN'
  TLS/SSL connection has been closed (EOF) (_ssl.c:997)

Traceback:
  File "~/.local/lib/python3.10/site-packages/offlineimap/accounts.py", line 293, in syncrunner
    self.__sync()
  File "~/.local/lib/python3.10/site-packages/offlineimap/accounts.py", line 372, in __sync
    remoterepos.getfolders()
  File "~/.local/lib/python3.10/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "~/.local/lib/python3.10/site-packages/offlineimap/imapserver.py", line 557, in acquireconnection
    imapobj = imaplibutil.WrappedIMAP4_SSL(
  File "~/.local/lib/python3.10/site-packages/offlineimap/imaplibutil.py", line 202, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 2191, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 360, in __init__
    self.open(host, port)
  File "~/.local/lib/python3.10/site-packages/offlineimap/imaplibutil.py", line 210, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 2204, in open
    self.ssl_wrap_socket()
  File "~/.local/lib/python3.10/site-packages/offlineimap/bundled_imaplib2.py", line 554, in ssl_wrap_socket
    self.sock = ctx.wrap_socket(self.sock, server_hostname=self.host)
  File "/usr/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()

Steps to reproduce the error

SSL notes

The server uses TLS version 1. The standard OpenSSL configuration in Ubuntu 22.04 does not allow the protocol. To get something useful, I use this openssl config (grabbed from stackoverflow):

openssl_conf = default_conf

[ default_conf ]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
MinProtocol = TLSv1
CipherString = DEFAULT:@SECLEVEL=1

Looking up the SSL information on the host (OPENSSL_CONF=openssl_tls1.cnf openssl s_client -connect imap.cern.ch:993):

CONNECTED(00000003)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Organization Validation Secure Server CA
verify return:1
depth=0 C = CH, ST = Gen\C3\A8ve, O = CERN Organisation Europ\C3\A9enne pour la Recherche Nucl\C3\A9aire, CN = mmm.cern.ch
verify return:1
4067FB04837F0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:../ssl/statem/statem_clnt.c:2248:
---
Certificate chain
 0 s:C = CH, ST = Gen\C3\A8ve, O = CERN Organisation Europ\C3\A9enne pour la Recherche Nucl\C3\A9aire, CN = mmm.cern.ch
   i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Organization Validation Secure Server CA
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov 17 00:00:00 2021 GMT; NotAfter: Nov 17 23:59:59 2022 GMT
 1 s:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Organization Validation Secure Server CA
   i:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA384
   v:NotBefore: Nov  2 00:00:00 2018 GMT; NotAfter: Dec 31 23:59:59 2030 GMT
 2 s:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
   i:C = GB, ST = Greater Manchester, L = Salford, O = Comodo CA Limited, CN = AAA Certificate Services
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA384
   v:NotBefore: Mar 12 00:00:00 2019 GMT; NotAfter: Dec 31 23:59:59 2028 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHPDCCBiSgAwIBAgIRAJvljDCuAv8xD1aQG/TfxecwDQYJKoZIhvcNAQELBQAw
gZUxCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO
BgNVBAcTB1NhbGZvcmQxGDAWBgNVBAoTD1NlY3RpZ28gTGltaXRlZDE9MDsGA1UE
AxM0U2VjdGlnbyBSU0EgT3JnYW5pemF0aW9uIFZhbGlkYXRpb24gU2VjdXJlIFNl
cnZlciBDQTAeFw0yMTExMTcwMDAwMDBaFw0yMjExMTcyMzU5NTlaMHoxCzAJBgNV
BAYTAkNIMRAwDgYDVQQIDAdHZW7DqHZlMUMwQQYDVQQKDDpDRVJOIE9yZ2FuaXNh
dGlvbiBFdXJvcMOpZW5uZSBwb3VyIGxhIFJlY2hlcmNoZSBOdWNsw6lhaXJlMRQw
EgYDVQQDEwttbW0uY2Vybi5jaDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAK2FIE0gI0ElE4J1JgkAaDFUyDOKnRWxuiDtBHNKjGGiRuThoZD6zGIBHUNU
sYl/3OHaPWzyaksBIbpXF68QV4TOIGlcHte2x7teYJ2l/3VGe/G66ubk0fQeN1pR
mSo52MKrNZY6EM90vD4fV464m69FivhZgq5AqHdO1RF+8x6WfkCL6EkBbfp9tswi
/FSWPVkyVOt8UlxDBTHNDXnRpyl3aysaZWt6QPZDvvDX8XpHpfby7jEzYgwKha8S
+Ojr6tYve+3bivRIuPCIFv34qv/l5/BTS928XYuqWsRNzp5VFy1mggAxSyBOR/7L
dGgWtykm/P+VtCuuxu8KBqDUH50CAwEAAaOCA58wggObMB8GA1UdIwQYMBaAFBfZ
1iUnZ/kxwklD2TA2RIxsqU/rMB0GA1UdDgQWBBQLwS6tYjWxWP8h6PNyphiXe/95
SjAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEF
BQcDAQYIKwYBBQUHAwIwSgYDVR0gBEMwQTA1BgwrBgEEAbIxAQIBAwQwJTAjBggr
BgEFBQcCARYXaHR0cHM6Ly9zZWN0aWdvLmNvbS9DUFMwCAYGZ4EMAQICMFoGA1Ud
HwRTMFEwT6BNoEuGSWh0dHA6Ly9jcmwuc2VjdGlnby5jb20vU2VjdGlnb1JTQU9y
Z2FuaXphdGlvblZhbGlkYXRpb25TZWN1cmVTZXJ2ZXJDQS5jcmwwgYoGCCsGAQUF
BwEBBH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0
aWdvUlNBT3JnYW5pemF0aW9uVmFsaWRhdGlvblNlY3VyZVNlcnZlckNBLmNydDAj
BggrBgEFBQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wggGABgorBgEEAdZ5
AgQCBIIBcASCAWwBagB2AEalVet1+pEgMLWiiWn0830RLEF0vv1JuIWr8vxw/m1H
AAABfS3s9EgAAAQDAEcwRQIgCTT2BXq1qv+Mk5nOBw1mwDYPLvrxHMo5JJroN9KO
Y/ACIQDNijd4Zr7n1tWpfC9lY2EH2C35kcEwEFd90RsV4ZV3VwB3AEHIyrHfIkZK
EMahOglCh15OMYsbA+vrS8do8JBilgb2AAABfS3s9AcAAAQDAEgwRgIhALVRLDVU
GZNG8k004kdcQBfQKlQOinbPtfEFe/uKu1rtAiEA9qLNSfGuNsyNIHBCqH4eQIf8
Zgft9PM4zkelvBL7+6AAdwApeb7wnjk5IfBWc59jpXflvld9nGAK+PlNXSZcJV3H
hAAAAX0t7PPbAAAEAwBIMEYCIQDCrZ1/17YmWbyWJTsj7CDiQZGFFsNkjiq3dkWN
h2MmSAIhANJbY3b4BraKTbygghU/OHvIQxGbDsg8IKeu8HMFoeEWMGMGA1UdEQRc
MFqCC21tbS5jZXJuLmNoghRhdXRvZGlzY292ZXIuY2Vybi5jaIIMaW1hcC5jZXJu
LmNoggxsZGFwLmNlcm4uY2iCC3BvcC5jZXJuLmNoggxzbXRwLmNlcm4uY2gwDQYJ
KoZIhvcNAQELBQADggEBADaIonx0sd5nn58VGO8T4ePUuwpbDcA9jqgXFd7nkWpd
O7TUrVKIE+GeTRGgOxkMrQJSuwOc4RA85q1/r9aWiPyjaSrkQH90x2mKNaowBjkD
a8rovVEdPatJ6xw2w9ZXdCwsc3qx0o3FkyD4kUy2Qb3Mfh2f0efrZ4+UpZv6wS6V
WiKLpQPBUdheeANI7j4BUfPs7PcVFNqPazmNh+fn26lr9kyIWC0sIPCUbH1Fs8w2
0xY3WLGtG+tcHyDjHuxoUQ5fKhYMn4kDAsNaCtZo6ZJpZEf+M4Be0ZraYRlqoWbM
EKQE5avfy/KpwnAShgG6izVhyMxMX+yID5BGQIpaklE=
-----END CERTIFICATE-----
subject=C = CH, ST = Gen\C3\A8ve, O = CERN Organisation Europ\C3\A9enne pour la Recherche Nucl\C3\A9aire, CN = mmm.cern.ch
issuer=C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Organization Validation Secure Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 5275 bytes and written 325 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID: F0440000F344B0FAADA4582FF9E4850D271ECADAAE8521B4165FA965651AF1F3
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1651566890
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---

Input that openssl config blurb into the /etc/ssl/openssl.cnf, but that did not change the error when running offlineimap.

What am I doing wrong?

berghaus commented 2 years ago

Workaround: run in pyton2 with openssl 1.1:

OfflineIMAP 7.3.4 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) imaplib2 v2.101 (bundled), Python v2.7.18, OpenSSL 1.1.1d 10 Sep 2019

I did this by using a container, which I built like so:

FROM python:2-alpine
RUN apk add openssl & pip install offlineimap==7.3.4
ENTRYPOINT ["offlineimap"]

and run like so

function offlineimap {
    if [[ -z "${XDG_CONFIG_HOME}" ]]; then
        CONFIG="${HOME}/"
    else
        CONFIG="${XDG_CONFIG_HOME}/offlineimap/config"
    fi
    docker run \
        --user "${UID}:${GID}" \
        --mount type=bind,source="${CONFIG}",target="/.offlineimaprc" \
        --volume "${HOME}/Mail":"/Mail" \
        --volume "${HOME}/.offlineimap":"/.offlineimap" \
        --rm -it \
        offlineimap "$@"
}

Where my local accounts end up in ~/Mail, but show up in the config now as /Mail ... could probably make that cleaner.

igsha commented 8 months ago

I had a similar problem. I needed to pass ciphers parameter into ssl:

OfflineIMAP 8.0.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v3.06, Python v3.11.6, OpenSSL 3.0.13 30 Jan 2024
Account sync main:
 *** Processing account main
 Establishing connection to XXXXXX:993 (main-remote)
 ERROR: Unknown SSL protocol connecting to host 'XXXXXX' for repository 'main-remote'. OpenSSL responded:
[SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:1006)
 *** Finished account 'main' in 0:05
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'XXXXXX' for repository 'main-remote'. OpenSSL responded:
[SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:1006)

Traceback:
  File "/nix/store/i6wdj5klim3cwh5cxb1apc3vjaaqxv7y-offlineimap-8.0.0/lib/python3.11/site-packages/offlineimap/accounts.py", line 298, in syncrunner
    self.__sync()
  File "/nix/store/i6wdj5klim3cwh5cxb1apc3vjaaqxv7y-offlineimap-8.0.0/lib/python3.11/site-packages/offlineimap/accounts.py", line 374, in __sync
    remoterepos.getfolders()
  File "/nix/store/i6wdj5klim3cwh5cxb1apc3vjaaqxv7y-offlineimap-8.0.0/lib/python3.11/site-packages/offlineimap/repository/IMAP.py", line 681, in getfolders
    imapobj = self.imapserver.acquireconnection()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/i6wdj5klim3cwh5cxb1apc3vjaaqxv7y-offlineimap-8.0.0/lib/python3.11/site-packages/offlineimap/imapserver.py", line 663, in acquireconnection
    raise OfflineImapError(reason, severity, exc_info()[2])

So my solution was to add transporttunnel into config's remote section:

[Repository main-remote]
type = IMAP
remotehost = XXXXXX
remoteuser = <login>
remotepasseval = get_pass("main@XXXXXX")
auth_mechanisms = LOGIN
transporttunnel = openssl s_client -tls1 -cipher ALL:@SECLEVEL=0 -connect XXXXXX:993 -quiet

This approach works fine with python3.