simonrob / email-oauth2-proxy

An IMAP/POP/SMTP proxy that transparently adds OAuth 2.0 authentication for email clients that don't support this method.
Apache License 2.0
828 stars 89 forks source link

Outlook 5.7.3 Authentication unsuccessful #297

Open betacatgo opened 5 days ago

betacatgo commented 5 days ago

Since Microsoft no longer allows basic authentication after September 16th, 2024 (app passwords don't work either), I could no longer use git-send-email to send patches through my Outlook personal account, and then I found this project.

5.7.3 Authentication unsuccessful [LO4P123CA0207.GBRP123.PROD.OUTLOOK.COM 2024-10-10T21:50:33.251Z 08DCE9686F3D146F]

After a long time of struggling with the configuration, I am still stuck on this error and I think I need some help.

./pyenv/bin/python3 emailproxy.py --no-gui --local-server-auth --debug
2024-10-10 23:49:45: Initialising Email OAuth 2.0 Proxy (version 2024-10-04) in debug mode from config file /home/xxx/email-oauth2-proxy/emailproxy.config
2024-10-10 23:49:45: Starting IMAP server at 127.0.0.1:1993 (unsecured) proxying outlook.office365.com:993 (SSL/TLS)
2024-10-10 23:49:45: Starting POP server at 127.0.0.1:1995 (unsecured) proxying outlook.office365.com:995 (SSL/TLS)
2024-10-10 23:49:45: Starting SMTP server at 127.0.0.1:1587 (unsecured) proxying smtp.office365.com:587 (STARTTLS)
2024-10-10 23:49:45: Initialised Email OAuth 2.0 Proxy - listening for authentication requests. Connect your email client to begin
2024-10-10 23:49:56: New incoming connection to SMTP server at 127.0.0.1:1587 (unsecured) proxying smtp.office365.com:587 (STARTTLS)
2024-10-10 23:49:56: Accepting new connection from 127.0.0.1:53878 to SMTP server at 127.0.0.1:1587 (unsecured) proxying smtp.office365.com:587 (STARTTLS)
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) --> [ Client connected ]
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'220 LO4P302CA0005.outlook.office365.com Microsoft ESMTP MAIL Service ready at Thu, 10 Oct 2024 22:49:55 +0000 [08DCE8B62813AB2B]\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) <-- b'220 LO4P302CA0005.outlook.office365.com Microsoft ESMTP MAIL Service ready at Thu, 10 Oct 2024 22:49:55 +0000 [08DCE8B62813AB2B]\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) --> b'EHLO debian.aux.lan\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     --> b'EHLO debian.aux.lan\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-LO4P302CA0005.outlook.office365.com Hello [193.115.217.23]\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-DSN\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-STARTTLS\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     --> b'STARTTLS\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'220 2.0.0 SMTP server ready\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) <-> [ Starting TLS handshake ]
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) [ Successfully negotiated SMTP server STARTTLS connection - re-sending greeting ]
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     --> b'EHLO debian.aux.lan\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) <-> [ TLSv1.3 handshake complete ]
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-LO4P302CA0005.outlook.office365.com Hello [193.115.217.23]\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-SIZE 157286400\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-PIPELINING\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-DSN\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-ENHANCEDSTATUSCODES\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-AUTH LOGIN XOAUTH2\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-8BITMIME\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-BINARYMIME\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250-CHUNKING\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'250 SMTPUTF8\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) <-- b'250-LO4P302CA0005.outlook.office365.com Hello [193.115.217.23]\r\n250-SIZE 157286400\r\n250-PIPELINING\r\n250-DSN\r\n250-ENHANCEDSTATUSCODES\r\n250-AUTH PLAIN LOGIN\r\n250-8BITMIME\r\n250-BINARYMIME\r\n250-CHUNKING\r\n250 SMTPUTF8\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) --> b'AUTH PLAIN [[ Credentials removed from proxy log ]]\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     --> b'AUTH XOAUTH2\r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587)     <-- b'334 \r\n'
2024-10-10 23:49:56: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587; xxx.xxx@outlook.com)     --> b'[[ Credentials removed from proxy log ]]\r\n'
2024-10-10 23:50:01: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587; xxx.xxx@outlook.com)     <-- b'535 5.7.3 Authentication unsuccessful [LO4P302CA0005.GBRP302.PROD.OUTLOOK.COM 2024-10-10T22:50:01.030Z 08DCE8B62813AB2B]\r\n'
2024-10-10 23:50:01: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587; xxx.xxx@outlook.com) <-- b'535 5.7.3 Authentication unsuccessful [LO4P302CA0005.GBRP302.PROD.OUTLOOK.COM 2024-10-10T22:50:01.030Z 08DCE8B62813AB2B]\r\n'
2024-10-10 23:50:01: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587) <-- [ Server disconnected ]
2024-10-10 23:50:01: SMTP (127.0.0.1:53878-{127.0.0.1:1587}-smtp.office365.com:587; xxx.xxx@outlook.com) --> [ Client disconnected ]

Above is the failed debug information that frustrates me.

[user]
    email = xxx.xxx@outlook.com
    name = XXX XXX
[sendemail]
    smtpServer = 127.0.0.1
    smtpUser = xxx.xxx@outlook.com
    smtpPass = xxxxx
    smtpServerPort = 1587
    confirm = always
    suppresscc = all

Above is my gitconfig.

[IMAP-1993]
server_address = outlook.office365.com
server_port = 993
local_address = 127.0.0.1

[POP-1995]
server_address = outlook.office365.com
server_port = 995
local_address = 127.0.0.1

[SMTP-1587]
server_address = smtp.office365.com
server_port = 587
server_starttls = True
local_address = 127.0.0.1

[xxx.xxx@outlook.com]
permission_url = https://login.microsoftonline.com/tenant id/oauth2/v2.0/authorize
token_url = https://login.microsoftonline.com/tenant id/oauth2/v2.0/token
oauth2_scope = https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/POP.AccessAsUser.All https://outlook.office.com/SMTP.Send offline_access
redirect_uri = http://localhost:8080
client_id = xxxxxxxxxxxx
client_secret = xxxxxxxxxxxx
token_salt = xxxxxxxx
token_iterations = 870000
access_token = xxxxxxxxxxxxxxx
access_token_expiry = 1728601964
refresh_token = xxxxxxxxxxx

[emailproxy]
delete_account_token_on_password_error = True
encrypt_client_secret_on_first_use = False
use_login_password_as_client_credentials_secret = False
allow_catch_all_accounts = False

Above is my emailproxy.config

I can already get the token_salt, access_token, refresh_token, but the authentication is still unsuccessful.

azure

azure2

azure3

azure4

azure5

azure6

azure7

Above are all my configurations in Azure.

auth

auth2

I have completed the permission acceptance and can show OAuth 2.0 proxy successfully.

I am sure that SMTP is not disabled in my Outlook as I can use Thunderbird to send emails successfully.

I have tried changing smtp.office365.com to smtp-mail.outlook.com, https://outlook.office.com/SMTP.Send to https://graph.microsoft.com/SMTP.Send, offline_access to https://graph.microsoft.com/offline_access and Web application to SPA application, but none of them work.

I have tried everything I can think of.

If anyone can help me I would be very grateful.

This is important to me.

Many thanks!

simonrob commented 4 days ago

Thanks for the detailed report. Unfortunately I don't have capacity to troubleshoot Azure/Entra setups, but there are plenty of other guides to help you navigate the confusing process required - this was the first result for me just now, for example.

Re: proxy setup, you should stick with what is in the example configuration file (i.e., not the Graph scopes).

betacatgo commented 4 days ago

Thanks for your reply, I tried the method in the article you mentioned, but it doesn't work.

In fact, I have tried many articles on configuring Azure/Entra and none of them work.

The following is some additional information.

security

As a personal account, there are actually no Office 365 permissions, and only Graph permissions are related to emails.

security2

security3

I have tried turning off all the security configurations I can find, but it doesn't work.

security4

security5

security6

I have also tried turning off two-step authentication, or turning on two-step authentication and using app passwords, but neither works.

I struggled for a long time here, but couldn't make the authentication successful.

If there is still no solution, I can only give up my Outlook account and use other email providers...

simonrob commented 4 days ago

Ah, that's an important detail - if you're using a free Outlook account you'll need to reuse an OAuth client ID that has been approved by Microsoft as you're not able to approve your own (you're not the administrator). There are links in the proxy's readme to various options here.

betacatgo commented 3 days ago

Thanks for the information, it helped me finally find the cause.

Since you mentioned that I need to use a Microsoft approved client id, it made me curious to find out what client id Thunderbird uses.

After some time of debugging using the Thunderbird Developer Tool, I found it all at OAuth2Providers.sys.mjs.

The Thunderbird client id can also be found in this blog.

Outlook personal accounts may not require complex Azure/Entra configurations, as Thunderbird client id can be used (interestingly, we don't need to provide client_secret when using Thunderbird client id).

But when I use the Thunderbird client id, the same error appears again.

5.7.3 Authentication unsuccessful.

This makes me suspect that this is not the problem.

thunderbird

proxy

I compared the authentication process of Thunderbird with email-oauth2-proxy which is exactly the same but with different results.

thunderbird2

Eventually I debugged Thunderbird with breakpoints and I discovered that the OAuthToken sent by Thunderbird was very different from the one sent by email-oauth2-proxy.

The OAuthToken length sent by Thunderbird is 1585, but the length sent by email-oauth2-proxy is 3401.

After I base64 decoded it, only the very beginning user=xxx.xxx@outlook.comauth=Bearer is the same, while the rest is very different.

proxy2

proxy3

I tried to modify the code in SMTPOAuth2ServerConnection and replace OAuth2Helper.encode_oauth2_string(result) with the OAuthToken I got in Thunderbird.

I finally saw the long-awaited 250 OK and the email was sent successfully!

250 2.0.0 OK <XXXXX@XXXXX.eurprd03.prod.outlook.com>

I am not an email expert, but can confirm that there should be bugs in OAuth2Helper.get_oauth2_credentials or OAuth2Helper.encode_oauth2_string.

Hopefully this information provided above can help you fix it.

Many thanks!