Could not connect via SSL to host 'xxx' and non-standard ssl port 143 configured.

[hydhyd]

General informations

Debian 8 / Linux xxx 2.6.32-042stab120.11 #1 SMP Wed Nov 16 12:05:45 XXX 2016 x86_64 GNU/Linux offlineimap v7.1.1, imaplib2 v2.57 (bundled), Python v2.7.9, OpenSSL 1.0.1t 3 May 2016

Configuration file offlineimaprc

[general] accounts = xxx

[Account xxx] localrepository = Local remoterepository = Remote

[Repository Local] type = Maildir localfolders = ~/xxx sync_deletes = no

[Repository Remote] type = IMAP expunge = no starttls = yes readonly = True remoteport = 143 remotehost = xxx remoteuser = xxx remotepass = xxx folderfilter = lambda folder: folder not in ['Calendar'] sslcacertfile = /etc/ssl/certs/ca-certificates.crt

Logs, error

OfflineIMAP 7.1.1 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) imaplib2 v2.57 (bundled), Python v2.7.9, OpenSSL 1.0.1t 3 May 2016 Account sync xxx: Processing account xxx Establishing connection to xxx:143 (Remote) ERROR: Could not connect via SSL to host 'xxx' and non-standard ssl port 143 configured. Make sure you connect to the correct port. Finished account 'xxx' in 0:00 ERROR: Exceptions occurred during the run! ERROR: Could not connect via SSL to host 'xxx' and non-standard ssl port 143 configured. Make sure you connect to the correct port.

Traceback: File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 281, in syncrunner self.sync() File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 347, in sync remoterepos.getfolders() File "/root/offlineimap-7.1.1/offlineimap/repository/IMAP.py", line 452, in getfolders imapobj = self.imapserver.acquireconnection() File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 629, in acquireconnection exc_info()[2]) File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 544, in acquireconnection af=self.af, File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 194, in init super(WrappedIMAP4_SSL, self).init(*args, **kwargs) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2183, in init IMAP4.init(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 361, in init self.open(host, port) File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 202, in open super(WrappedIMAP4_SSL, self).open(host, port) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2196, in open self.ssl_wrap_socket() File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version) File "/usr/lib/python2.7/ssl.py", line 891, in wrap_socket ciphers=ciphers) File "/usr/lib/python2.7/ssl.py", line 566, in init self.do_handshake() File "/usr/lib/python2.7/ssl.py", line 788, in do_handshake self._sslobj.do_handshake()

Steps to reproduce the error

The remote IMAP server is a MS Exchange running on port 143 and absolutely requiring STARTTLS pre-auth. offlineimap 6.3.4 had no issues connecting whatsoever (provided, startttls = yes was explicitely specified) with this very same config, however it got stuck on that one message in INBOX:

IMAP server 'Remote' does not have a message with UID '40456'

Since copy_ignore_eval (http://www.offlineimap.org/configuration/2016/07/18/ignoring-UIDs.html) doesn't exist in 6.x, I upgraded to the latest 7.1.1 off the git tree. Now it seemingly doesn't like to STARTTLS on a non-standard port. Added the sslcacertfile option to the config as some have suggested to no avail.

I must add/stress that requiring STARTTLS pre-auth doesn't mean the connection must OR should be initialized in SSL mode, hence port 143. We're going SSL after the connection has been made.

Herein I believe lies the answer. OfflineIMAP is expecting SSL straight off the bat, but it doesn't happen by design.

What do?

Thanks.

[nicolas33]

Could you please run offlineimap with -dimap for IMAP debug mode?

[nicolas33]

And try with ssl = no, please.

[hydhyd]

Sure thing.

offlineimap 7.1.1:

root@xxx:~/offlineimap-7.1.1# ./offlineimap.py -dimap OfflineIMAP 7.1.1 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) imaplib2 v2.57 (bundled), Python v2.7.9, OpenSSL 1.0.1t 3 May 2016 Debug mode: Forcing to singlethreaded. Now debugging for imap: IMAP protocol debugging Now debugging for : Other offlineimap related sync messages Account sync Xxx: [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN'] Processing account Xxx Establishing connection to xxx:143 (Remote) [imap]: Remote: level 'tls_compat', version 'None' ERROR: Could not connect via SSL to host 'xxx' and non-standard ssl port 143 configured. Make sure you connect to the correct port. [' File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 281, in syncrunner\n self.sync()\n', ' File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 347, in sync\n remoterepos.getfolders()\n', ' File "/root/offlineimap-7.1.1/offlineimap/repository/IMAP.py", line 452, in getfolders\n imapobj = self.imapserver.acquireconnection()\n', ' File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 629, in acquireconnection\n exc_info()[2])\n', ' File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 544, in acquireconnection\n af=self.af,\n', ' File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 194, in init\n super(WrappedIMAP4_SSL, self).init(args, kwargs)\n', ' File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2183, in init\n IMAP4.init(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', ' File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 361, in init\n self.open(host, port)\n', ' File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 202, in open\n super(WrappedIMAP4_SSL, self).open(host, port)\n', ' File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2196, in open\n self.ssl_wrap_socket()\n', ' File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket\n self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)\n', ' File "/usr/lib/python2.7/ssl.py", line 891, in wrap_socket\n ciphers=ciphers)\n', ' File "/usr/lib/python2.7/ssl.py", line 566, in init\n self.do_handshake()\n', ' File "/usr/lib/python2.7/ssl.py", line 788, in do_handshake\n self._sslobj.do_handshake()\n'] *** Finished account 'xxx' in 0:00 ERROR: Exceptions occurred during the run! ERROR: Could not connect via SSL to host 'xxx' and non-standard ssl port 143 configured. Make sure you connect to the correct port.

Traceback: File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 281, in syncrunner self.sync() File "/root/offlineimap-7.1.1/offlineimap/accounts.py", line 347, in sync remoterepos.getfolders() File "/root/offlineimap-7.1.1/offlineimap/repository/IMAP.py", line 452, in getfolders imapobj = self.imapserver.acquireconnection() File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 629, in acquireconnection exc_info()[2]) File "/root/offlineimap-7.1.1/offlineimap/imapserver.py", line 544, in acquireconnection af=self.af, File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 194, in init super(WrappedIMAP4_SSL, self).init(*args, **kwargs) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2183, in init IMAP4.init(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 361, in init self.open(host, port) File "/root/offlineimap-7.1.1/offlineimap/imaplibutil.py", line 202, in open super(WrappedIMAP4_SSL, self).open(host, port) File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 2196, in open self.ssl_wrap_socket() File "/root/offlineimap-7.1.1/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version) File "/usr/lib/python2.7/ssl.py", line 891, in wrap_socket ciphers=ciphers) File "/usr/lib/python2.7/ssl.py", line 566, in init self.do_handshake() File "/usr/lib/python2.7/ssl.py", line 788, in do_handshake self._sslobj.do_handshake()

offlineimap 6.3.4 (from Debian repository)

, the relevant bit: DEBUG[imap]: Returned object from fetching 40589: 'Received: from XXX.COM (1.2.3.4) by XXX (1.2.3.4) with Microsoft SMTP Server (TLS) id 14.0.722.0; Fr...ieE9BBdYxNC87ye8JKIf8SEe6nSSJBxx5rWcsZ+SwAAAAAAAA ' XXX reader: DEBUG[imap]: 07:03.36 XXX reader poll => [] DEBUG[imap]: 07:33.36 XXX reader poll => [] DEBUG[imap]: 08:03.39 XXX reader poll => [] DEBUG[imap]: 08:33.42 XXX reader poll => [] DEBUG[imap]: 09:03.46 XXX reader poll => [] ^C ^\Quit [ad infinitum]

[hydhyd]

nicolas33, you're a freaking lifesaver.

offlineimap 7.1.1 with "ssl = no" works. I mean, it connects, with "starttls = on" still in it. Let's see if it ignores the offending UID.

Thanks a bunch. Will update whichever way it goes further. From the looks of it, it's going alright but kinda dont wanna jinx it.

Should we put this in the FAQ? I believe it belongs in the FAQ, seeing as how many people are experiencing issues with the MS Exchange. Except I don't know what I just did, except probably telling it not to go SSL straight away.

It's going smoothly as we speak and it's past the goddamned UID, from what I can tell.

[nicolas33]

Thanks. I've improved the doc in the case users need to force STARTTLS.

Will update whatever way it goes further.

Ok. Then, I'll let you close this issue when it's fine on your side.

[hydhyd]

Again, thanks a bunch, you saved me hours if not days of frustration.

Since I'm up at 6am anyway, I might as well see this thing through. Will close the ticket when/if the job is done. You feedback was indispensable.

Seems to be running smoothly, so I guess the issue has been settled. For those unfortunate enough to deal with MS Exchange's weird configuration, here's the config file that worked for me (YMMV):

[general] accounts = Xxx

[Account Xxx] localrepository = Local remoterepository = Remote

[Repository Local] type = Maildir localfolders = ~/Xxx sync_deletes = no

[Repository Remote] type = IMAP expunge = no starttls = yes ssl = no readonly = True remoteport = 143 remotehost = xxx.host remoteuser = xxx.user remotepass = xxx.password folderfilter = lambda folder: folder not in ['Calendar'] sslcacertfile = /etc/ssl/certs/ca-certificates.crt copy_ignore_eval = lambda foldername: {'INBOX': [UID_THAT_OIMAP_DIES_ON_YOU]}.get(foldername)

Closing the ticket for now. Again, thanks for the marvellous support provided.

[nicolas33]

I couldn't sleep this night and decided to wake up at 4am. So, it's a little bit of luck. :-)

http://www.offlineimap.org/doc/be-effective.html

Happy to see it works. ,-)

[hydhyd]

I woke up at 3am, so that might be the cosmos doing its cosmos thing.

Can I ask one last thing though?

Do the messages get flagged as "Seen" as per the RFC after being downloaded with OI or do they stay as they were? I have not had the opportunity to check.

If so, any switch to not alter their state?

[nicolas33]

The flags remain the same and we currently do not support marking emails as seen as soon as they are downloaded. This is expected behaviour in IMAP mode. This feature might worth, though. Feel free to open a new issue about that if you feel the need for this feature. No promise this will be implemented. It's the first time I'm hearing this request.

As a workaround, you might write a script to mark downloaded emails as seen once on disk and run it as a post sync hook. They will be marked seen on next sync.

[hydhyd]

In fact, I'm very happy with them not being marked as seen since it would probably involve all kinds of conflicts between different proper email clients that me or someone else might use on a day-to-day basis with the same account.

This here is a backup solution, after all.

I personally don't feel this is needed, unless properly documented with a possibility to switch it off.

Just wanted to be sure, since I only started using OI. And it's marvelous.

Wishing you all guys would get together and figure a complete solution akin to fetchmail+hypermail to crontab the hell out of some IMAP account publishing on the web in a fairly pretty way. I'm talking https://github.com/RaymiiOrg/NoPriv or rather https://github.com/polo2ro/imapbox.

Neither of those can process offline Maildirs. And you're really good at the online part.

That's all wishful thinking anyway. Love your project.

[nicolas33]

Ok, so we're fine with the current behaviour.

If it's for backups, please read http://www.offlineimap.org/doc/backups.html.

PS: I like the cosmos. :-)

[hydhyd]

I realize the point of your project and it's darned good at what it does.

That's more me rambling at how we're very limited at publishing Maildir/mbox data on the web since that kinda went more off vogue.

[nicolas33]

Oh, and you might like to add readonly = True on the remote section. By adding sync_deletes = no too, the backups will be more conservative.

All the details are in the configuration file.

I don't know if offlineimap is darned good, actually. We have open issues especially with Microsoft products. You might like to extend the filtered folders: http://www.offlineimap.org/doc/FAQ.html#imap-server-notes.

[hydhyd]

Oh, you're damn sure I filtered them Calendar/Notes folders. Redacted them off for the sake of privacy in the ticket. From what I understand, Outlook alone knows how to deal with them.

A shame, too, seeing as how that would potentially be useful. Converting Calendar to .ical, Notes to .txt. That's just me daydreaming.

The important thing is that the meat (the actual mail) is taken off the bone ever so thoroughly. That's what matters in the end.

Every IMAP project on github has open issues with MS Exchange (and believe me, I went through about all of them). It seems you just can't escape it. Never noticed the pre-defined list of those "can't sync" folders. Definitely a nice touch, but with every account comes a bit of fine-tuning. I wish I could just blacklist "Calendar/*".

The options you suggested don't really apply to my situation, from what I understand from the docs since I'm only mirroring the remote mailbox. Still, you got me thinking I might want to read the documentation /s.

[nicolas33]

I agree. And yes, it's kind of sad IMAP got twisted like that without standards.