Outlook: 535 5.7.3 Authentication unsuccessful

[kpedro88]

My org switched to OAuth2 for everything recently. They provided example code for programmatic mail access, which works for me. (A copy of that code, with internal details and secrets redacted, can be found at https://www.dropbox.com/s/5ci49ha472su1hi/oauth2-samples.zip?dl=0.)

I've hacked around the (very nice!) codebase here to make a few changes (base64 encoding, some additional settings, etc.) at https://github.com/kpedro88/sasl-xoauth2/tree/base64. I'm now sure that this code and my org's example are sending the exact same encoded token message when given the same content in the token file (access secret, expiry, refresh secret). Despite that, sasl-xoauth2 still can't authenticate.

A few observations that may point toward the problem:

1. The internal docs for our org's registered client app say "No secret is provided, use the client credential flow option."
2. The flow to obtain credentials in our org's code is different than from sasl-xoauth2. The org code uses Microsoft's msal library in Python. When requesting a new token, the prompt is:

   To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXXXX to authenticate. Then in the web browser, the login dialog shows: You're signing in to [org app] on another device located in United States. and once logged in, lands on https://login.microsoftonline.com/appverify with the message: You have signed in to the [org app] application on your device. You may now close this window.

3. This alternative flow with msal doesn't allow specifying 'offline_access' in the user-defined scope, but does produce a refreshable token.

I don't fully understand the difference between this flow and the one that sasl-xoauth2 is using, but I suspect it may be the source of the problem.

Unfortunately, I am not aware of a C++ implementation of msal. Is it possible to implement a sasl plugin in Python? (I'm not very familiar with sasl plugin specifications.)

Other suggestions or corrections also welcome, of course.

[tarickb]

Hey, thanks for the detailed testing notes and pointers here. I did some experimentation with my consumer Outlook account (that's all I have access to) and I can confirm that the instructions in the sasl-xoauth2 README are still valid, and that Outlook will accept an OAuth2 token that is not Base64-encoded. I did discover that the offline_access part of the token scope is indeed critical -- without that I kept getting 535 5.7.3 Authentication unsuccessful messages from Outlook.

Can you try requesting a token using sasl-xoauth2-tool? You'll have to tweak the endpoint in the config file, and you may or may not need to specify a client secret, but if you can get past that, you should have a token usable with Postfix.

At some point this week/weekend I'll try committing an update to sasl-xoauth2-tool that uses the (much simpler) device flow for login, but that's more of a usability improvement than anything else.

[kpedro88]

I had previously tried using sasl-xoauth2-tool (our client app has no secret); I get the token, and it passes the refresh test sasl-xoauth2-tool test-token-refresh, but postfix authentication fails with the token when trying to relay email (same error message 535).

I'm not sure about whether base64 is actually necessary; our internal docs point to https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#sasl-xoauth2, which says that it is.

I also noticed that our docs say to use outlook.office365.com instead of smtp.office365.com as the SMTP server address, but I've tested both and it doesn't seem to make a difference.

I have not been able discern any other obvious differences between the flows so far.

[tarickb]

Okay so as of 0.23, you can use the device flow to request tokens for Outlook. I've updated the instructions in the README but basically you need to enable "Allow public client flows" for your app registration (in the Azure console), and pass --use-device-flow to sasl-xoauth2-tool.

Can you give that a try and see if you're able to authenticate over SMTP with Postfix?

[kpedro88]

@tarickb I noticed the commits, thanks! I tried it out earlier today, and I can get a token just fine, but somehow postfix still won't authenticate when sending an email. I have a few more differential changes to try, but it seems like it must be something very minor about my setup that's misconfigured somehow.

Also, you may want to add in the readme that when installing from source (sudo) pip(3) install msal will be needed as a prerequisite. (Not sure how this works in the prepackaged Ubuntu version.)

[tarickb]

Done, thanks for the heads-up! I've also added a config option for /etc/sasl-xoauth2.conf named always_log_to_syslog that will generate some diagnostics in syslog, in case that comes in handy. Feel free to share your Postfix config files (with sensitive details redacted of course) if you want a second set of eyes.

[kpedro88]

It works now! I had outsmarted myself - I was still using my patched version that applied base64 encoding manually. It turns out that was harmful, because postfix sasl already applies it by default: https://github.com/vdukhovni/postfix/blob/0abb45ca0732fc039e0f43b66ed441d283eb2565/postfix/src/xsasl/xsasl_cyrus_client.c#L483. (I found this difficult to discern from postfix's documentation - might be worth noting it somewhere in case others also read MS' documentation and get worried...) So in the end, it was just the device flow that I needed - our org's app only supports that, as far as I can tell.

(Edit to add: I eventually figured this out by enabling full verbosity from postfix, following https://www.postfix.org/DEBUG_README.html#verbose for the smtp client in master.cf. Just noting this here for posterity...)

I picked out the still-relevant parts of my branch, now here: https://github.com/kpedro88/sasl-xoauth2/tree/user

• fix a minor mistake in the readme, pointed out in a previous issue
• option to specify username manually (our org uses some email/account name aliasing, so this is a necessary feature for me)
• avoid truncation in logging output

It would be great to get these upstreamed - let me know if you have any concerns or if I should just send a PR.

Thanks again for your help!

[tarickb]

That's great, I'm glad you got it figured out! I added a pointer to the Postfix debug page to the README. As for your changes -- do feel free to send a PR. I can take the positional-argument fix, but for the others: 1) I think the user parameter makes more sense in a token file rather than in the config file, since some deployments have more than one user, and 2) I can't accept code that's been copied-and-pasted from elsewhere (but I'll work on a similar change to the logging code myself).

[kpedro88]

1. I agree, and I've reimplemented it that way.
2. Looks like it's already done now - thanks!

[kpedro88]

see #74

[tarickb]

Thank you!