msgraph DeviceCode auth reports error AADSTS7000218 despite providing client_secret

[jdghub]

I have upgraded to ParseDMARC 8.3.0 to use msgraph authentication instead of IMAP, but UsernamePassword doesn't seem to work (because the account has MFA enabled) and ClientSecret was getting complicated (because the account was created as a shared mailbox, though direct login is now enabled). So I would like to use DeviceCode authentication. I've:

• registered ParseDMARC as as an app
• recorded the Application (client) ID and the Directory (tenant) ID
• assigned the Mail.ReadWrite delegated permission
• created and recorded a Client Secret for the registered app

My INI file has [msgraph] auth_method = DeviceCode tenant_id = <tenant-id> client_id = <client-id> client_secret = <client-secret-value> mailbox = <mbox>@<domain>

When I run ParseDMARC I'm prompted to login with a code, which I do as <user>@<domain>, including MFA, and I accept to access ParseDMARC. But when I close the login window ParseDMARC exits with a long error stream that starts DeviceCodeCredential.get_token failed: Authentication failed: AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'. Trace ID: b0378a2c-560e-44a2-8420-141397fcbe00 Correlation ID: e1c56f5d-c66d-48d8-8b47-717d151b205b Timestamp: 2022-06-30 05:52:13Z Content: {"error":"invalid_client","error_description":"AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.\r\nTrace ID: b0378a2c-560e-44a2-8420-141397fcbe00\r\nCorrelation ID: e1c56f5d-c66d-48d8-8b47-717d151b205b\r\nTimestamp: 2022-06-30 05:52:13Z","error_codes":[7000218],"timestamp":"2022-06-30 05:52:13Z","trace_id":"b0378a2c-560e-44a2-8420-141397fcbe00","correlation_id":"e1c56f5d-c66d-48d8-8b47-717d151b205b","error_uri":"https://login.microsoftonline.com/error?code=7000218"} and then repeats 2 variants of the same info.

My INI file provides the client_secret value, and ParseDMARC errors out if it's not there. So is ParseDMARC not providing it for some reason, or is there something else I might have missed?

Thanks.

[nathanthorpe]

To use the Device code flow you must have the Allow public flows option enabled

[jdghub]

Thanks for this. I'll check/test and report back.

[jdghub]

Allowing public client flows for the app (under Authentication) allows ParseDMARC to access the mailbox and process the contents:

DEBUG:__init__.py:1092:Found 10 messages in Inbox
DEBUG:__init__.py:1100:Processing 10 messages
DEBUG:__init__.py:1104:Processing message 1 of 10: UID <...>
 <...>
DEBUG:__init__.py:1104:Processing message 10 of 10: UID <...>

But it then fails with:

DeviceCodeCredential.get_token failed: Interactive authentication is required to get a token. Call 'authenticate' to begin.
Content: {"error":"invalid_grant",
  "error_description":"AADSTS50196: 
    The server terminated an operation because it encountered a client request loop. 
    Please contact your app vendor.\r\n
    Trace ID: <id>\r\n
    Correlation ID: <id>\r\n
    Timestamp: 2022-07-20 21:17:35Z",
  "error_codes":[50196],
  "timestamp":"2022-07-20 21:17:35Z",
  "trace_id":"<id>",
  "correlation_id":"<id>",
  "error_uri":"https://login.microsoftonline.com/error?code=50196"}
ERROR:cli.py:945
<repeat of the same error>
Traceback (most recent call last):
<...>

Looking for info on AADSTS50196 I found suggestions that it can be related to cached info in a browser, so I ran ParseDMARC again and this time authenticated to the mailbox in a Chrome incognito window, but got the same error.

Any suggestions?

Thanks.

[jdghub]

Also: I just noticed that there are actually 533 emails in the DMARC reports Inbox waiting to be processed, not just the 10 reported by ParseDMARC above Why would it see only 10?

[nathanthorpe]

Oh yeah I think we may have to implement token cache. Did not realize this since I only tested it one time. I use the ClientCredentials flow :smile:.

For your second question, yeah that is because the graph API has a default limit, see related #333

[jdghub]

Thanks, I just saw #333 now.

I'm using Device Code flow because the DMARC mailbox started life as a shared mailbox, and while logins were subsequently enabled it can't be used for ClientCredentials flow in its current state. But presumably the token cache is required for all flows?

[jdghub]

And #319 reminds me that the real reason I'm using DeviceCode here is that MFA is enabled on all accounts, and Conditional Access isn't available with Business Standard...

[nathanthorpe]

Token cache is only really used in the DeviceCode flow since that is the only one that prompts you to re-log in. ClientCredentials does work for shared mailboxes though if you are interested.

[jdghub]

I was unable to use ClientCredentials via direct logon with MFA to the shared mailbox (originally enabled to allow IMAP access.) Will it work via authentication (also with MFA) to a delegate account?

[nathanthorpe]

In the client credentials flow you don't need to log in as a user (or use MFA) because you are authenticating as an application rather than a user.

You'd need to grant it application permission in the app registration rather than delegated permission for it to work.

Also you will need to restrict the access to the single mailbox since the app has access to all by default. There is a command to run that is in the readme.

[jdghub]

Thanks, but I see that I'd mixed up the history on what I'd tried here. You're right, it was UsernamePassword flow that failed due to MFA.

I then tried ClientSecret flow, but that failed because this mailbox was created as a shared mailbox (though I later enabled logons to allow direct IMAP access) and the mailbox was not accepted in the New-ApplicationAccessPolicy command because a shared mailbox cannot be a security principal.

At that point I thought it might be quicker to configure DeviceCode flow than de-sharify the mailbox.

Since you've bundled the batch size fix (needed regardless of flow) with the token cache fix, hopefully it will be easiest for me to stick with DeviceCode flow here.

[jdghub]

Let me know if this is an issue with my environment rather than ParseDMARC, but I'm getting an error when I update to the latest commit here (while waiting for a new version on PyPI.)

I'm running ParseDMARC in a portable install of Python 3.10 under Windows, and originally installed ParseDMARC 8.3.0 from PyPI. I just uninstalled this via pip uninstall and then installed the latest commit via

python.exe -m pip install --no-warn-script-location https://github.com/domainaware/parsedmarc/archive/b15425f50e50637baa2b1181f6ad9c4399538e0d.tar.gz

This works, but when I run this version I get an error

> python3\scripts\parsedmarc.exe -c .\pd.ini
    INFO:cli.py:757:Starting parsedmarc
0it [00:00, ?it/s]
   ERROR:cli.py:904:MS Graph Error: No module named 'pywintypes'
Traceback (most recent call last):
  File "X:\Scripts\ParseDMARC\Python3\Lib\site-packages\parsedmarc\cli.py", line 892, in _main
    mailbox_connection = MSGraphConnection(
[...]
  File "X:\Scripts\ParseDMARC\Python3\Lib\site-packages\portalocker\portalocker.py", line 16, in <module>
    import pywintypes
ModuleNotFoundError: No module named 'pywintypes'`

Uninstalling this ParseDMARC version and reinstalling via PyPI solves the problem and ParseDMARC runs, but without the latest MSGraph fixes.

Thanks.

[nathanthorpe]

does manually installing this work pip install pypiwin32?

[jdghub]

No, it sees it as already there:

Requirement already satisfied: pypiwin32 in ...scripts\parsedmarc\python3\lib\site-packages (223)
Requirement already satisfied: pywin32>=223 in ...\scripts\parsedmarc\python3\lib\site-packages (from pypiwin32) (304)

Which is true - the DLLs are where they should be, neither uninstalling nor reinstalling ParseDMARC shows any change to any dependencies. The install shows:

Collecting https://github.com/domainaware/parsedmarc/archive/b15425f50e50637baa2b1181f6ad9c4399538e0d.tar.gz
  Using cached https://github.com/domainaware/parsedmarc/archive/b15425f50e50637baa2b1181f6ad9c4399538e0d.tar.gz
  Preparing metadata (setup.py) ... done
[Requirement already satisfied: for all dependencies...]
Building wheels for collected packages: parsedmarc
  Building wheel for parsedmarc (setup.py) ... done
  Created wheel for parsedmarc: filename=parsedmarc-8.3.0-py3-none-any.whl size=3658008 sha256=d6a7bc3e6a856bb0732e1ca9fd336679ba16286f62d3c327ba9d754f800ec97f
Stored in directory: C:\Users\[user]\AppData\Local\Temp\pip-ephem-wheel-cache-p95gh2p0\wheels\77\7b\02\19e00ee0358d536edd8ce1b23a3533334462b7bc60430ea838
Successfully built parsedmarc
Installing collected packages: parsedmarc
Successfully installed parsedmarc-8.3.0

Yet something changes which prevents the module (or the DLLs) from being located at runtime. Uninstalling and reinstalling from PyPI (also cached) fixes it.

[jdghub]

I spent a lot of time trying to find out what was different between the PyPI and github installs that caused this runtime error: ModuleNotFoundError: No module named 'pywintypes' But in the end I resolved it by giving up on portable Python and installed the standard 3.10.6 x64 installer.