search

simonrob / email-oauth2-proxy

An IMAP/POP/SMTP proxy that transparently adds OAuth 2.0 authentication for email clients that don't support this method.
Apache License 2.0
850 stars 95 forks source link

Issues with "STARTTLS not supported and the user requires enforcement., code: 500" #169

Closed mshlis closed 1 year ago

mshlis commented 1 year ago

Setup:

Returns: "STARTTLS not supported and the user requires enforcement., code: 500"

In Debug Mode, I get the following outputs:

2023-06-08 16:25:57: New incoming connection to SMTP server at :1587 (unsecured) proxying smtp.office365.com:587 (STARTTLS)
2023-06-08 16:25:57: Accepting new connection to SMTP server at :1587 (unsecured) proxying smtp.office365.com:587 (STARTTLS) via ('::ffff:209.85.216.46', 39430, 0, 0)
2023-06-08 16:25:57: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) --> [ Client connected ]
2023-06-08 16:25:58: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'220 CH0PR04CA0006.outlook.office365.com Microsoft ESMTP MAIL Service ready at Thu, 8 Jun 2023 20:25:57 +0000\r\n'
2023-06-08 16:25:58: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'220 CH0PR04CA0006.outlook.office365.com Microsoft ESMTP MAIL Service ready at Thu, 8 Jun 2023 20:25:57 +0000\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) --> b'EHLO mail-pj1-f46.google.com\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     --> b'EHLO mail-pj1-f46.google.com\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-CH0PR04CA0006.outlook.office365.com Hello [75.135.101.210]\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-DSN\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-STARTTLS\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     --> b'STARTTLS\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'220 2.0.0 SMTP server ready\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-> [ Starting TLS handshake ]
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) [ Successfully negotiated SMTP STARTTLS connection - re-sending greeting ]
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     --> b'EHLO mail-pj1-f46.google.com\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-> [ TLSv1.3 handshake complete ]
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-CH0PR04CA0006.outlook.office365.com Hello [75.135.101.210]\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-CH0PR04CA0006.outlook.office365.com Hello [75.135.101.210]\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-SIZE 157286400\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-PIPELINING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-DSN\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-DSN\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-AUTH LOGIN XOAUTH2\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-AUTH PLAIN LOGIN\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-8BITMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-BINARYMIME\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250-CHUNKING\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- b'250 SMTPUTF8\r\n'
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) --> [ Client disconnected ]
2023-06-08 16:25:59: SMTP (:1587; ::ffff:209.85.216.46:39430->smtp.office365.com:587) <-- [ Server disconnected ]

But not sure how to interpret it/ see the error. Verified credentials and keys.

simonrob commented 1 year ago

The proxy log doesn't show any issues or errors. The log also shows a successful STARTTLS connection, but it is worth noting that this is handled entirely by the proxy, and Gmail will not have any knowledge about this.

To clarify, where are you seeing this message – in the Gmail interface? Issue #9 discusses a working setup for Gmail forwarding – it might be worth trying one of the other ports Gmail supports perhaps.

mshlis commented 1 year ago

So the response is from gmail, when in settings using the proxy server for adding an alias (in the add acounts section)

I am trying to reproduce the success in issue #9 that you referenced and switching over to ssl. I adjusted my current SMTP section settings to be:

[SMTP-587]
server_address = smtp.office365.com
server_port = 587
local_certificate_path = /home/ec2-user/mkcert/ec2-3-80-62-142.compute-1.amazonaws.com+4.pem
local_key_path = /home/ec2-user/mkcert/ec2-3-80-62-142.compute-1.amazonaws.com+4-key.pem
starttls = True

I generated certs from mkcert and also followed issue #14 and used the [Install Certificates.command](https://gist.github.com/marschhuynh/31c9375fc34a3e20c2d3b9eb8131d8f3#file-install-certificates-command) as it has assisted others with similar issues

but now getting a certs issue: "TLS Negotiation failed, the certificate doesn't match the host., code: 550"

note that getting same lack of error in debug:

2023-06-09 20:08:52: New incoming connection to SMTP server at :587 (TLS) proxying smtp.office365.com:587 (STARTTLS)
2023-06-09 20:08:52: Accepting new connection to SMTP server at :587 (TLS) proxying smtp.office365.com:587 (STARTTLS) via ('209.85.216.54', 55202)
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-> [ Starting TLS handshake ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) --> [ Client connected ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'220 MN2PR01CA0015.outlook.office365.com Microsoft ESMTP MAIL Service ready at Fri, 9 Jun 2023 20:08:51 +0000\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'220 MN2PR01CA0015.outlook.office365.com Microsoft ESMTP MAIL Service ready at Fri, 9 Jun 2023 20:08:51 +0000\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-> [ TLSv1.3 handshake complete ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) --> b'EHLO mail-pj1-f54.google.com\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     --> b'EHLO mail-pj1-f54.google.com\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-MN2PR01CA0015.outlook.office365.com Hello [3.80.62.142]\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-DSN\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-STARTTLS\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     --> b'STARTTLS\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'220 2.0.0 SMTP server ready\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-> [ Starting TLS handshake ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) [ Successfully negotiated SMTP STARTTLS connection - re-sending greeting ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     --> b'EHLO mail-pj1-f54.google.com\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-> [ TLSv1.3 handshake complete ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-MN2PR01CA0015.outlook.office365.com Hello [3.80.62.142]\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-MN2PR01CA0015.outlook.office365.com Hello [3.80.62.142]\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-SIZE 157286400\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-PIPELINING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-DSN\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-DSN\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-ENHANCEDSTATUSCODES\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-AUTH LOGIN XOAUTH2\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-AUTH PLAIN LOGIN\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-8BITMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-BINARYMIME\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250-CHUNKING\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- b'250 SMTPUTF8\r\n'
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) --> [ Client disconnected ]
2023-06-09 20:08:52: SMTP (:587; 209.85.216.54:55202->smtp.office365.com:587) <-- [ Server disconnected ]
simonrob commented 1 year ago

The mkcert tool is for local certificates; you'll need to use something like LetsEncrypt to work with Gmail. Try that and see whether it gets you any further?

mshlis commented 1 year ago

Fixing that does take let the requests actually go through!

Now the error is:

b'535 5.7.3 Authentication unsuccessful [BL0PR01CA0035.prod.exchangelabs.com 2023-06-11T23:01:05.961Z 08DB6A52DC3AB4FC]\r\n'

I verified credentials, and also took advice from similar closed issues on this repo (added offline permissions, changed app-side redirect to http://localhost, etc) but still no luck. Any idea?

simonrob commented 1 year ago

It's hard to debug these AAD issues, and it's not something the proxy has much control over. Personally I'd always start with the simplest possible configuration: the basic SMTP server, local client usage (i.e., not from Gmail), and a client ID/secret from an existing client (see the examples in the readme). Other than that, I'm afraid I can't really offer much insight into what is wrong here.

mshlis commented 1 year ago

I tried both but instead of 5.7.8 i get 5.7.3 invalid credentials for that one and im assuming its because the email is a tenant within an org and it hard-stops it if i try authenticate via basic configuration

please feel free to close the issue, I understand this is probably out of scope of (a) the proxy and (b) this issues initial purpose, but I really do appreciate the support + if it does manage to work in the end will potentially be a valuable use-case for others

simonrob commented 1 year ago

Thanks for following up. I'll do as you suggest and close this issue for now, but if you do find a resolution it would be great to hear how you solve this – I'd be glad to point others towards a fix too.