Crash with OpenSSL 1.1.1e and Gmail certificates

[milouse]

General informations

I know the bug is not in offlineimap code, but surely in the ssl implementation of python2. But I create the ticket to keep this problem in mind as a lot of people will begin to have it. This ticket may help to find a workaround.

Also, this problem seems to only affect Gmail account, or maybe only XOAUTH2 config. I cannot confirm that for now.

• system/distribution (with version): Archlinux up-to-date
• offlineimap version (offlineimap -V):
  • offlineimap v7.3.2
  • imaplib2 v2.101 (bundled)
  • Python v2.7.17
  • OpenSSL 1.1.1e 17 Mar 2020
• server name or domain: Gmail
• CLI options: none

Configuration file offlineimaprc

[Repository <snip>]
type = Gmail
auth_mechanisms = XOAUTH2
remoteuser = <snip>
realdelete = no
ssl = yes
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
ssl_version = ssl23
folderfilter = lambda foldername: not re.search('^(?:(?:\[Gmail\]/(?:Corbeille|Important|Spam|Suivis)$)|Errors)', foldername)
oauth2_client_id = <snip>
oauth2_client_secret = <snip>
oauth2_refresh_token = <snip>
oauth2_request_url = https://accounts.google.com/o/oauth2/token

Logs, error

OfflineIMAP 7.3.2
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.101 (bundled), Python v2.7.17, OpenSSL 1.1.1e  17 Mar 2020
Account sync <snip> :
*** Processing account <snip>
 Establishing connection to imap.gmail.com:993 (<snip>)
 ERROR: While attempting to sync account '<snip>'
  (u"[SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:1946) (configuration is: {'client_secret': '<snip>', 'grant_type': 'refresh_token', 'refresh_token': '<snip>', 'client_id': '<snip>'})",)
 *** Finished account '<snip>' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account '<snip>'
  (u"[SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:1946) (configuration is: {'client_secret': '<snip>', 'grant_type': 'refresh_token', 'refresh_token': '<snip>', 'client_id': '<snip>'})",)

Traceback:
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 293, in syncrunner
    self.__sync()
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 369, in __sync
    remoterepos.getfolders()
  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 586, in acquireconnection
    self.__authn_helper(imapobj)
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 450, in __authn_helper
    if func(imapobj):
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 384, in __authn_xoauth2
    imapobj.authenticate('XOAUTH2', self.__xoauth2handler)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 742, in authenticate
    typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 1748, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 1474, in _command
    literal = literator(data, rqb)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2339, in process
    ret = self.mech(self.decode(data))
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 257, in __xoauth2handler
    six.reraise(type(e), type(e)(msg), exc_info()[2])
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 251, in __xoauth2handler
    self.oauth2_request_url, urllib.urlencode(params)).read()
  File "/usr/lib/python2.7/socket.py", line 355, in read
    data = self._sock.recv(rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 754, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 641, in read
    v = self._sslobj.read(len)

Steps to reproduce the error

$ offlineimap -a <AnyGmailAccount> -o

[dkim286]

I'm having the same problem. Can't even generate new tokens using oauth2.py either:

$ python2.7 oauth2.py --user=<x> --client_id=<x> --client_secret=<x> --generate_oauth2_token
To authorize token, visit this url and follow the directions:
  https://accounts.google.com/o/oauth2/auth?client_id=<x>
Enter verification code: <x>
Traceback (most recent call last):
  File "oauth2.py", line 347, in <module>
    main(sys.argv)
  File "oauth2.py", line 326, in main
    authorization_code)
  File "oauth2.py", line 214, in AuthorizeTokens
    response = urllib.urlopen(request_url, urllib.urlencode(params)).read()
  File "/usr/lib/python2.7/socket.py", line 355, in read
    data = self._sock.recv(rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 754, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 641, in read
    v = self._sslobj.read(len)
ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:1946)

Maybe something changed on Google's end?

[trsdln]

As temporal solution you can downgrade openssl to previous version: sudo pacman -U /var/cache/pacman/pkg/openssl-1.1.1.d-2-x86_64.pkg.tar.xz

[dkim286]

Thank you, that seemed to have fixed it for now.

[bobek]

I am facing the same issue, reported to Debian as Bug954491. To downgrade openssl (which "fixes" the problem as @trsdln mentioned already), you need to

cd /var/cache/apt/archives && sudo dpkg -i openssl_1.1.1d-2_amd64.deb libssl1.1_1.1.1d-2_amd64.deb libssl1.1_1.1.1d-2_i386.deb

[J0nnyMak0]

openssl updates usually fix vulnerabilities. Downgrading openssl should not be recommended.

[roussosalex]

In this case OpenSSL changed the behavior on EOF in 1.1.1e causing various problems: https://github.com/openssl/openssl/issues/11381

[J0nnyMak0]

FYI

https://github.com/openssl/openssl/pull/11400/commits/6611672ca5fe3aa97a50aa9092f190e07879ed8c

[milouse]

Fixed by https://github.com/openssl/openssl/releases/tag/OpenSSL_1_1_1f

[nicolas33]

Great news. Thanks for the report!