Enhancement: Add support for logging in from managed devices when access policy "block access from apps on unmanaged devices" is active

mbuergi commented 3 years ago

Description

With the SharePoint Admin Center Policy "Block access from apps on unmanaged devices" enabled, users are unable to login to M365 CLI, even when using an ADJoined Device.

Login fails for all federated users. Login is successful for cloud-only accounts (onmicrosoft accounts).

Steps to reproduce

Login to https://tenantname-admin.sharepoint.com and enable the Access control policy for unmanaged devices.
This automatically creates an Azure AD Conditional Access Policy.
Login to M365 CLI with "M365 Login" from a managed device
Open https://microsoft.com/devicelogin in a browser and enter the code provided by M365 Shell, click next
Enter email Address in the "Sign in" Window, click next
"...taking you to your organizations sign-in page" pops up briefly.
Message appears: Help us keep your device secure Your sign-in was successful but your admin requires the device requesting access to be managed by 'tenant-name' to access this resource.

Expected result

Successful sign in with the user provided during the sign-in dialog.

Actual result

Unable to Sign in. Below the troubleshooting details, in the last line it says "Device State: DomainJoined"

When clicking okay, the browser is redirected to: https://login.microsoftonline.com/common/oauth2/nativeclient?error=access_denied&error_subcode=cancel

The logs show, that the conditional access policy created in the first step above blocked the login.

Environment

M365 Version: v3.3.0 OS: Windows 10 Version 1909

We use AD FS, users are authenticated onPrem.

Thanks for lookting into this Matt

plamber commented 3 years ago

Thank you @mbuergi

We will check internally and come back to you asap

waldekmastykarz commented 3 years ago

Following the same setup, can you sign in using device code auth for example in Azure CLI or PnP PowerShell?

mbuergi commented 3 years ago

Following the same setup, can you sign in using device code auth for example in Azure CLI or PnP PowerShell?

PnP PowerShell: Logging in with "Connect-PnPOnline -Url https://tenant-name.sharepoint.com -PnPManagementShell" I have the exact same behaviour. But there are many other options to login with PnPManagementShell that are working for me.

Azure CLI Logging in with "az login" I am directly redirected to https://login.microsoftonline.com/common/oauth2/authorize?response_type=code&client_id=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxx&redirect_uri=http%3a%2f%2flocalhost%3a8400&state=xxxxxxxxx&resource=https%3a%2f%2fmanagement.core.windows.net%2f&prompt=select_account&sso_nonce=xxxxxxxxxxxx&client-request-id=xxxxxxxxxxxxxxxxxxxxx&mscrid=xxxxxxxxxxxxxxxxxx

After entering/confirming my email address in the sign-in box, I am automatically signed in. The following message is then displayed in the shell (before all subscriptions are listed):

You have logged in. Now let us find all the subscriptions to which you have access... Failed to authenticate '{'additional_properties': {}, 'id': '/tenants/b4c9f32e-da17-4ded-9c95-xxxxxxxxxxxx', 'tenant_id': 'b4c9f32e-da17-4ded-9c95-xxxxxxxxxxxx', 'tenant_category': <TenantCategory.home: 'Home'>, 'country': None, 'country_code': 'US', 'display_name': None, 'domains': ['msftcommunity.com', 'msftcommunity.onmicrosoft.com']}' due to error 'Get Token request returned http error: 400 and server response: {"error":"interaction_required","error_description":"AADSTS53003: Access has been blocked by Conditional Access policies. The access policy does not allow token issuance.\r\nTrace ID: 5d00a54f-b617-41cf-89d9-fa78ff9f3802\r\nCorrelation ID: e9af60d8-24a2-40aa-a439-c1825d235b93\r\nTimestamp: 2020-12-01 15:24:21Z","error_codes":[53003],"timestamp":"2020-12-01 15:24:21Z","trace_id":"5d00a54f-b617-41cf-89d9-fa78ff9f3802","correlation_id":"e9af60d8-24a2-40aa-a439-c1825d235b93","error_uri":"https://login.microsoftonline.com/error?code=53003","suberror":"message_only"}' The following tenants don't contain accessible subscriptions. Use 'az login --allow-no-subscriptions' to have tenant level access. 2ef93d8f-f806-4ea0-95d6-xxxxxxxxxxxx 37a5cd44-f1af-4568-8217-xxxxxxxxxxxx

The tenant-Ids in this message are unknown to me... Despite this message, I am logged in to az CLI and can successfully execute commands.

I don't know how to sign in with device code authentication. If there is a way and you want me to test it, please advise

garrytrinder commented 3 years ago

I've not come across this issue myself however it looks like this is specific to using device code flow and conditional access policies, rather than a specific issue with CLI as you are seeing similar behaviour on other tools.

The below article suggests that this is by design.

device code flow could be completed on another device like a smartphone or a computer than the device which initially initiated the sign-in / authorization process

So it’s by design that the device code flow cannot satisfy any device-based conditional access rules. Furthermore, device code flow falls into the “Unknown” client application section. The Azure AD identity platform simply doesn’t know if you’re signin-in for an app on your smart TV, IOT device or within PowerShell and about the device state.

Source: https://tech.nicolonsky.ch/device-code-auth-ca/

What login options work for you in PnP PowerShell?

mbuergi commented 3 years ago

What login options work for you in PnP PowerShell?

On managed device, interactive session, i use either Connect-PnPOnline -CurrentCredentials or Connect-PnPOnline -UseWebLogin

For scheduled scripts, running on a server which is not hybrid joined, I have registered an App and authenticate with AppID and AppSecret.

waldekmastykarz commented 3 years ago

Like @garrytrinder mentioned, since other tools show similar behavior, I'd say this is by design. What's interesting though is, if there is another way we could support authentication that would work in your scenario.

I'd suggest that we rephrase this issue and change it from a bug into a feature request. Agreed?

mbuergi commented 3 years ago

I'd suggest that we rephrase this issue and change it from a bug into a feature request. Agreed?

Sure - I leave that up to you guys how you want to handle this.

plamber commented 3 years ago

Hi @mbuergi, we will verify how to integrate an interactive authentication to the CLI.

To implement it we might need to verify how the azure cli fixes this issue. We need to integrate the interactive authentication through a browser in the solution. The challenge we are facing is that we want to implement the solution in a way to be cross-platform.

We will follow these basic steps:

verify if the azure cli interactive authentication works on a setup like yours
verify how the interactive authentication of azure cli is implemented
integrate this type of authentication into our cli

br, Patrick

mbuergi commented 3 years ago

Hi @plamber Thanks for the update - let me what you need me to do in order to verify the azure cli interactive authentication on our setup. regards Matt

plamber commented 3 years ago

Hi all, I have a short summary of all the findings related to this case.

Azure CLI The Azure CLI opens tries to open the browser of the user and authenticates the user. If it fails it follows the device code flow logic we already implemented in our CLI. This might be a nice user experience for our CLI authentication but does not seem to solve the issue. We found out that the Azure CLI has not a registered Enterprise Application in the tenant and does not follow the same conditional access policy restrictions our PnP Application is following. Therefore, the authentication with the Azure CLI runs without issues while the Pnp App fails with the device code flow.

Connect-PnPOnline To compare other solutions we verified how the Connect-PnPOnline Commandlet runs. It fails when using username or password all the times due to the conditional access policy.

The only solution that works with Connect-PnPOnline is the -useWebLogin (interactive login) option. This specific technique, however, is not cross-platform.

Next steps:

We will verify if we can exclude the PnP App from the conditional access rule to see if there are not some other issues related to this setup
We will verify if there is a way to implement an interactive login in node.js

plamber commented 3 years ago

@pnp/cli-for-microsoft-365-maintainers: I have some ideas that might require your input.

I thought we could use a technique like the one presented here to open a chromium session using Puppeteer. This session will open a site with an MSAL authentication that will ask the user to authenticate to Azure AD using an interactive login. With Puppeteer we will then retrieve the generated token and return it back to the CLI for further processing.

There are two possible approaches where the authentication site is hosted:

Somewhere on a CDN
Locally on node

Both options have pros and cons. I am still undecided which one might be the better one.

I am going to write a small PoC to see if this works at all. Afterwards, we can see how to proceed.

Maybe you have some better ideas how to tackle the interactive authentication problem?

waldekmastykarz commented 3 years ago

@plamber thanks for sharing your findings

We found out that the Azure CLI has not a registered Enterprise Application in the tenant and does not follow the same conditional access policy restrictions our PnP Application is following.

That is correct indeed. Azure CLI uses a 1st party AAD app which work under different rules than 3rd party apps like the one we use. Since CLI for Microsoft 365 is not a Microsoft product, it's unlikely that we'd get a 1st party app to use. We could fallback to using the Azure CLI AAD app but it doesn't have all API permissions which means that you couldn't use all CLI functionality with it.

The only solution that works with Connect-PnPOnline is the -useWebLogin (interactive login) option. This specific technique, however, is not cross-platform.

This is good point indeed. While the latest PnP PowerShell x-platform version offers this option, it works only on Windows. Personally, I wouldn't want us to add an option that works only on some platforms and not other.

With Puppeteer we will then retrieve the generated token and return it back to the CLI for further processing.

This seems like a hack to me that I'd personally be against implementing. I'd rather look into more robust approaches, especially since we're talking about auth. At this moment I don't have a better alternative, but I also haven't looked into what's possible yet. Perhaps we should spend more time investigating what's possible first.

plamber commented 3 years ago

@waldekmastykarz

After some more digging I can state that we are not able to support synchronized AD accounts that are secured with second factors. I think of admin accounts that are secured with more complex conditional access rules such as trusted devices. In bigger corporations definitely a must.

For these cases you require an interactive authentication that opens a browser and authenticates your user properly. When looking at PnP PowerShell the -UseWebLogin is the only one that allows a proper authentication of these types of secured accounts. The UseWebLogin does nothing else than open a browser and perform an appropriate authentication flow. It is done with a "BrowserHelper" class.

This means if we want to support such use cases we need a way to open a browser window on all platforms and retrieve the necessary tokens.

Any ideas which cross-platform library we could use to "talk" to the local browser of a device?

garrytrinder commented 3 years ago

The GitHub CLI seems to have solved this problem, during Auth setup, after showing a device code in the terminal, it asks you to hit return which opens your default browser, navigating to a login page where the device code is entered.

It's written in python but looks like it should be possible to open the default browser a navigate to a custom url.

plamber commented 3 years ago

Navigating alone will most likely not solve our problem. I am sure that we require a browser session performing the interactive authentication. The results of the authentication in this browser instance need to go back to our cli.

This is how the useweblogin session does it in the pnp powershell solution

waldekmastykarz commented 3 years ago

It seems like Azure CLI launches a local web server on https://localhost:8400 when calling az login and this is where the launched browser redirects to, to finish the auth flow. Here is their implementation of auth and which we should be able to mimic in a x-platform compatible way: https://github.com/Azure/azure-cli/blob/e8b8740173ab612ec51f0c24c1da1e0f834c2120/src/azure-cli-core/azure/cli/core/_profile.py#L1270

Looking at the code, you will see that az cli is trying to find an available port in the range of 8400-9000. In case you wonder, it's not needed to define all these ports as a reply URI. When using localhost as redirect URI, you can use any port. More info: https://docs.microsoft.com/answers/answers/48979/view.html

garrytrinder commented 3 years ago

We could use https://github.com/expressjs/express as our local web server and use https://github.com/sindresorhus/open to provide a way of opening the browser window.

waldekmastykarz commented 3 years ago

Express is way too bloated for what we need. I don't think we need anything more than starting an instance of http server.

garrytrinder commented 3 years ago

Good shout on using http server @waldekmastykarz no extra dependencies that way 👍🏻

waldekmastykarz commented 3 years ago

Express is a great option if you have routing, middleware, auth, etc. In our case though, all we need is to monitor an endpoint for incoming requests and show a basic response in the browser, which we should be able to achieve using the standard http server. And indeed, it allows us to avoid any extra dependencies.

plamber commented 3 years ago

Hi, I am going to take this up.

br,

plamber commented 3 years ago

@waldekmastykarz and @garrytrinder

I am currently implementing a first version of the authentication flow using an HTTP server. We have different experiences to choose from.

Option A

We start an HTTP server
We ask the user to open the browser and use a link such as http://localhost:8888
When navigating to that link it shows a page with a short instruction and a "login" button
The login button will trigger the authentication which will use a generated authorization link such as https://login.microsoftonline.com/1840b34e-6fe7-4a56-9ad3-0b7b58c49dc7/oauth2/authorize?response_type=code&client_id=f98c87e8-8b5e-41c7-9815-c97b10220d32&redirect_uri=http://localhost:8888&state=8b8c87f759adf&resource=https://graph.microsoft.com/.default&prompt=select_account
We listen to the response. We are also going to report in the browser if the operation was successful or not
We are showing an error message in the console and in the Browser if the operation was not successful. In the browser we show also the login button to give it a try again

Option B

We start an HTTP server
We ask the user to open the browser and use a link such as http://localhost:8888
We ask the user to open the browser and use a link such as https://login.microsoftonline.com/1840b34e-6fe7-4a56-9ad3-0b7b58c49dc7/oauth2/authorize?response_type=code&client_id=f98c87e8-8b5e-41c7-9815-c97b10220d32&redirect_uri=http://localhost:8888&state=8b8c87f759adf&resource=https://graph.microsoft.com/.default&prompt=select_account
We listen to the response. We are also going to report in the browser if the operation was successful or not
If there was an issue, the person has to trigger again the login operation

I am more for Option A. Feels more natural to me

plamber commented 3 years ago

I just came up with an option C.

This is an extension of option B opening automatically the defaultbrowser with open

If ope ails, the authorization link is sown in the console.

This is eve a better flow than A

What do you think?

plamber commented 3 years ago

I have some more updates. Implemented a basic project that follows Option C. You can check it out here. To run it in your environment just change the values here.

npm i
npm start

The solution uses the node-adal package to acquire a token once a code has been retrieved as the Azure CLI does here. Unfortunately, I still have an issue with this flow and I haven't figured out what is the key solution used by the Azure CLI team.

The code grant flow requires a secret or client assertion if you want to obtain a token. If you do not pass this information Azure returns the error message: AADSTS7000218: The request body must contain the following parameter: ‘client_assertion’ or ‘client_secret’

If we want to support this type of authentication with our multi-tenant app, then we have to look for alternatives. If we are good asking the community to register their own app, then we can finalize the solution and require a custom app with client_assertion (certificate).

The Azure CLI team uses the code grant but passes the value "None" to the function retrieving the token. You see it here. If you check the documentation of the Python library you can see that the client secret is not required for the function 'acquire_token_with_authorization_code'.

What I haven't figured out is what they do differently to avoid to use the client secret to obtain the desired token. I know it from other projects. Usually, if you have backends you always require a secret or client_assertion to obtain the necessary tokens. Otherwise, you have to move to a different type of authentication such as Implicit Grant Flow or the new Single-Page application flow.

If we do not find a solution for the code grant and we still want to support this authentication for our multi-tenant app, then I would go to my initial suggestion and solve the issue with a client side authentication using an SPA. Once the token is retrieved we will pass the values to the local node site. Not sure if we should use a library as MSAL Browser. I am already using it actively in projects. On the other hand, the refresh token handling is performed by the library, which might be a problem for our flow.

waldekmastykarz commented 3 years ago

I'd suggest that we try to build an experience similar to az cli with which people are already familiar. Since our client is public anyway, why not create a secret that we use with the local server? Isn't that what az cli is doing too?

plamber commented 3 years ago

As long we ensure we are not requesting any app permissions, this approach might work fine. The moment our public CLI app requests app permissions, we are imposing a security risk

Furthermore, we might require a certificate that doesnt expire soon

Let me know if this might work out

plamber commented 3 years ago

@waldekmastykarz: are we fine to generate a certificate that expires in a couple of years and that we ensure to never add application permissions to our permission list?

waldekmastykarz commented 3 years ago

are we fine to generate a certificate that expires in a couple of years

Yes, we should be able to do that. We should also put a reminder to renew it timely.

that we ensure to never add application permissions to our permission list

That's been the case already, but always good to keep this in mind and check it regularly.

waldekmastykarz commented 3 years ago

It's taken a bit longer than originally planned, but we've just released a preview version v3.6 with this option implemented. Once again thank you for bringing the limitation to our attention and helping us with addressing it 👏

pnp / cli-microsoft365