STARTTLS required for many email servers

[KeyMasterOfGozer]

Gmail, Office365 and other large email vendors require STARTTLS usually port 587 for sending emails. The implementation in croncoat doesn't support that method. I made some simple changes to MailBackend to handle that. It requires adding a new line to your .croncoat.ini file starttls=Yes

This project is not active, so I'm not sure if this will be read, but maybe if someone else has this problem , they can grab this file and it might help them.

I'm not sure if it's the same for everyone, but for me, my mailbackend.py file is in /usr/local/lib/python2.7/site-packages/croncoat/cc

mailbackend.py.zip

Matthias Kauer, if you want to include this, please do, or feel free to throw away my sloppy code and use your own. :)

Update: I just made a fork and proposed changes via a pull request. I think that might have been a better way to do this. Either way, good luck guys!

[MatthiasKauer]

Hi, thank you for getting in touch and for the nice words.

The project is still active in the sense that I depend on it for several processes that are crucial for me. I had not noticed the need for StartTLS yet. Chances are that my email provider is a bit more lenient than yours.

The changes you made look fine to me (I looked at https://github.com/MatthiasKauer/croncoat/pull/7 the zip would be identical, right?) . Nothing sloppy in my opinion. I'll test this a bit and see from there.

I'm wondering if StartTLS should simply be the default instead of configurable. What do you think about that?

Best regards, Matthias

[KeyMasterOfGozer]

The zip file contains the same thing that I sent as the pull request.

I tested it. The starttls method doesn't work for normal SSL/TLS smtp servers. I noticed this before when setting up SMTP servers of Thunderbird. It gives 3 security methods to choose from: ["none","SSL/TLS","STARTTLS"].

I'm not sure if those are the only 3 options, or if that is only what thunderbird supports.

I'm don't want to give the impression that I'm an expert on this material, and I only first looked at python 3 months ago, but I try to contribute to the community where I can.

Please let me know if I can do anything else, and trust me, it won't hurt my feelings if you can find a better way.

Thanks again, Mike

[MatthiasKauer]

I see. I just checked Thunderbird and configured StartTLS there. This feels a bit weird because I should then also need it for croncoat. Chances are that I was lazy / stupid and simply login without encryption on my VPS. I'll sort this out soon.

[MatthiasKauer]

Ok, looks like I was using SSL by default so far. As far as I understand SSL is more secure than STARTTLS (see https://serverfault.com/questions/523804/is-starttls-more-safe-than-tls-ssl)

Do you really need STARTTLS? Gmail also supports SSL, no? Note that you may have to configure another port.

I'm also looking for a way to test sending emails somehow. Do you know a fake server for STARTTLS and/or SSL?

Regards, Matthias

[KeyMasterOfGozer]

Unfortunately, no. GMail, Office365, and almost all other large servers have moved to STARTTLS and do not support SSL/TLS, which is why I started this process. From what I have researched, using port 465 and normal SSL for SMTP is considered deprecated. Believe me, I tried using port 465 and normal SSL first.

I don't understand the intricacies of it, but it seems this is where SMTP is moving.

If you need to test, you can make a free gmail account to test.

[MatthiasKauer]

Before I start, note that I'm mainly trying to understand how these things are working. It probably still makes sense to have StartTLS and maybe even allow unencrypted traffic for a tool like this.

Do you have any evidence for the large providers moving away from SSL? A gmail or office365 blog post maybe? I only found some that referred to old SSL versions.

FWIW, I could use a Gmail account with the following settings:

[Mail]
smtpserver=smtp.gmail.com
smtpport=465
user=me@gmail.com
pass=<mypass>
fromaddr=admin@<mynotebook>.de

I had to activate "allow less secure apps" in Gmail but I had to do the same to make Thunderbird work without OAuth. What have you been trying?

[MatthiasKauer]

I have merged your pull request now. I have then updated the config semantics a bit. There is now a security option for none, starttls and ssl. That should work for you, no? Can you test that and provide feedback before I begin figuring out the PyPI release procedure again?

[KeyMasterOfGozer]

That works great for me. I have a few fires to put out this morning, but I will get this tested today and let you know.

[KeyMasterOfGozer]

I was able to test and it seems to be working for me. By the way, I like your implementation better than mine.

I didn't know that you could "Allow Less Secure Apps" in GMail. That might come in handy if I am using a device that can't do STARTTLS. It's obvious that Google is trying to steer you away from port 465, though.

I've learned some things about Internet email from looking into this that I didn't know.

Here is a 1997 memo from Paul Hoffman of the Internet Mail Consortium. Reference C at the end talks about the IANA revoking port 465 for SMTPS use. https://tools.ietf.org/html/draft-hoffman-smtp-ssl-04

The second post down on this StackExchange is where I saw he history: https://stackoverflow.com/questions/15796530/what-is-the-difference-between-ports-465-and-587

Office365 set instructions specify only StartTLS can be used: https://support.office.com/en-us/article/POP-and-IMAP-settings-for-Outlook-Office-365-for-business-7fc677eb-2491-4cbc-8153-8e7113525f6c

Facebook article from 2014 about STARTTLS seems to be referenced a lot: https://www.facebook.com/notes/protect-the-graph/the-current-state-of-smtp-starttls-deployment/1453015901605223/

[MatthiasKauer]

Thank you for looking into this and also thank you for the enlightening articles. I have uploaded a new version to PyPI with our recent changes. pip install croncoat should now fetch version 0.4. It does so on my machine ;)