Refresh token for Office365

[olantwin]

Thanks for writing this great tool!

I'm having some issues getting Office365 to work properly. I can authorize successfully and using the token in aerc works well, but after some time it fails with the following message:

mailctl: getEmailAuth:
renewAccessToken: Nothing as refresh token argument
CallStack (from HasCallStack):
  error, called at lib/MailCtl/Authorization.hs:136:22 in mailctl-0.9.2-inplace:MailCtl.Authorization

Trying to get to the bottom of the issue, it seems like no refresh tokens are provided by microsoft, and the correct way to refresh the token would be to resend the original authorization request, which should succeed if within the validity period (see e.g. here: https://stackoverflow.com/questions/75332566/how-to-get-a-refresh-token-with-oauth-in-microsoft365 )

It's of course also more than possible that I misconfiguration something somewhere.

I'd appreciate any hints! Please let me know if you require any further information!

[pdobsan]

On Fri, Oct 27, 2023 at 03:03:55AM -0700, Oliver Lantwin wrote:

It's of course also more than possible that I misconfiguration something somewhere.

How could we tell if you don't provide your config files?

I don't have microsoft account but some users do so we might be able to suggest corrections for your config.

I'd appreciate any hints! Please let me know if you require any

Immediately after authorization check your credentials returned to see if the refresh token is indeed missing. Regardless how you store it (keyring or gpg encrypted file) it is the same piece of json. Here are ways to dump and format it:

secret-tool lookup mailctl ***@***.*** | jq
gpg -d ***@***.*** | jq

Send this information too.

[olantwin]

Apologies for the incomplete information. Thanks for the quick reply!

Immediately after authorization check your credentials returned to see if the refresh token is indeed missing. Regardless how you store it (keyring or gpg encrypted file) it is the same piece of json. Here are ways to dump and format it: secret-tool lookup mailctl @. | jq gpg -d @. | jq Send this information too.

{
  "access_token": "REDACTED",
  "email": {
    "unEmailAddress": "oliver.lantwin@cern.ch"
  },
  "exp_date": "2023-10-27 21:52 UTC",
  "expires_in": 4447,
  "refresh_token": null,
  "scope": "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send",
  "service": "microsoft",
  "token_type": "Bearer"
}

The access_token works, but the refresh_token is null (this is immediately after mailctl authorize. Now I'm using the gnome-keyring, but I already ran into this issue using gpg.

config.yaml

services_file: /home/olantwin/.config/mailctl/services.yaml

ring_store:
  exec: secret-tool
  args:
    - store
    - --label

ring_lookup:
  exec: secret-tool
  args:
    - lookup
    - mailctl

services.yaml

microsoft:
  tenant: "cern.ch"
  auth_endpoint: https://login.microsoftonline.com/common/oauth2/v2.0/authorize
  auth_http_method: GET
  auth_params_mode: query-string
  token_endpoint: https://login.microsoftonline.com/common/oauth2/v2.0/token
  token_http_method: POST
  token_params_mode: request-body-form
  redirect_uri: http://localhost:8080
  auth_scope: https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send
  tenant_id : "c80d3499-4a40-4a8c-986e-abce017d6b19"
  # Thunderbird client ID and secret
  client_id: "08162f7c-0fd2-4200-a84a-f25a4db0b584"
  client_secret: "TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82"

[pdobsan]

On Fri, Oct 27, 2023 at 01:55:37PM -0700, Oliver Lantwin wrote:

services.yaml

microsoft:
  tenant: "cern.ch"
  auth_endpoint: https://login.microsoftonline.com/common/oauth2/v2.0/authorize
  auth_http_method: GET
  auth_params_mode: query-string
  token_endpoint: https://login.microsoftonline.com/common/oauth2/v2.0/token
  token_http_method: POST
  token_params_mode: request-body-form
  redirect_uri: http://localhost:8080
  auth_scope: https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send
  tenant_id : "c80d3499-4a40-4a8c-986e-abce017d6b19"
  # Thunderbird client ID and secret
  client_id: "08162f7c-0fd2-4200-a84a-f25a4db0b584"
  client_secret: "TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82"

According to the Microsoft docs: "refresh_token only provided if offline_access scope was requested." So add offline_access too to the auth_scope: line. The template service file distributed with the program contains this but in your config it is missing.

You may also try to use the tenant's or tenant_id's value or organizations instead of common in the ..._endpoint: url-s to see if that make any difference.

Try all these variations and let me know which ones, if any, are working.

[olantwin]

Re-adding offline_access did the trick.

I removed it when initially adjusting the config, as our organisation's migration guide (mistakenly?) indicated that it was not a scope we were allowed to request. I forgot to go back through all the changes later once everything worked to figure out which were really necessary...

Thanks a lot for the help! Out of all the different CLI oauth2 options I've looked into, mailctl seems the nicest way of dealing with office365, especially now that it works so seamlessly with gnome-keyring!

[pablob127]

Are you using this with offlineimap by any chance? I am having the same problem, but my refresh_token is not null.

[olantwin]

Haven't switched to oama yet, but I was using mailctl with aerc.