[FEATURE] Add support for Azure DevOps Pipeline Managed Identity in Connect-PnpOnline

[kkazala]

Expected behavior

I am deploying my SPFx solution using Azure DevOps pipeline. I have a Service Connection defined in the project, and the SPN has the following API permissions (I'm using Sites.Selected but for the sake of troubleshooting I granted Sites.FullControl.All) :

In the pipeline, I'm executing PowerShell code using the Service Connection:

        - task: AzurePowerShell@5
          inputs:
            azureSubscription: "${{ parameters.serviceConnectionName }}"
            azurePowerShellVersion: LatestVersion
            ScriptType: 'InlineScript'
            Inline: |
              whoami
              Install-Module PnP.PowerShell -Scope "CurrentUser" -Verbose -AllowClobber -Force

              # TEST 1
              try{
                Connect-PnPOnline -ManagedIdentity -Url "https://$(Az_TenantName).sharepoint.com/sites/appcatalog/"
                Get-PnPConnection
              }
              catch{
                Write-Host "Error connecting to tenant app catalog"
                Write-Host $_.Exception.Message
              }
              # TEST 2
              try{
                $graphToken= Get-AzAccessToken -ResourceUrl "https://graph.microsoft.com/"
                Connect-PnPOnline -AccessToken  $graphToken -Url "https://$(Az_TenantName).sharepoint.com/sites/appcatalog/"
                Get-PnPConnection
              }
              catch{
                Write-Host "Error connecting to tenant app catalog"
                Write-Host $_.Exception.Message
              }

Actual behavior

TEST 1 Connect-PnPOnline -ManagedIdentity fails with the following error:

Error connecting to tenant app catalog
{"error":"invalid_request","error_description":"Identity not found"}

TEST 2 The Connect-PnPOnline -AccessToken $graphToken works.

Steps to reproduce behavior

1. Create a project in Azure DevOps
2. Create a Service Connection pointing to your Azure Subscrption
3. Go to "Project Settings" / "Service Connections". Choose the connection created in step 2 and click on "Manage Service Principal". Go to "API Permissions" and grant Sites.FullControl.All Application permissions for Microsoft Graph and SharePoint.
4. Create a pipeline using the steps presented above, replacing ${{ parameters.serviceConnectionName }} with your connection name and $(Az_TenantName) with your tenant name
5. Execute the pipeline

What is the version of the Cmdlet module you are running?

2.2.0

Which operating system/environment are you running PnP PowerShell on?

• [ ] Windows
• [ ] Linux
• [ ] MacOS
• [ ] Azure Cloud Shell
• [ ] Azure Functions
• [ ] Other : Azure DevOps MS agent, ubuntu-latest

[kkazala]

No, my bad, I created connection using service principal authentication 🤦‍♀️

and using Managed Identity is for self hosted agents anyway...

[kkazala]

Now that we can create Azure DevOps service connections using workload identity federation, it would be really useful to be able to call Connect-PnPOnline from AzurePowerShell@5 task and be able to execute within the context of the current identity.

Would you be open to this idea?

what works for me:

steps:
- task: AzurePowerShell@5
  name: ConnectPnpOnline
  inputs:
    azureSubscription: DEV_Connection
    azurePowerShellVersion: latestVersion
    ScriptType: InlineScript
    Inline: |
      $url = "https://$(tenantName).sharepoint.com"
      Write-Host "##[debug]Connecting to $url/sites/$(siteName)"

      Write-Host "##[group]Install/Import  PS modules"
      Install-Module PnP.PowerShell -Scope "CurrentUser" -Verbose -AllowClobber -Force
      Write-Host "##[endgroup]"

      try {
        $azAccessToken = Get-AzAccessToken -ResourceUrl $url
        $conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -AccessToken $azAccessToken.Token -ReturnConnection
        Write-Host "##[debug]Get-PnPConnection"
        Write-Host $conn.Url

        Write-Host "##[debug]Get-PnPWeb"
        $web = Get-PnPWeb -Connection $conn
        Write-Host  $web.Title
      }
      catch {
          Write-Host "##[error] 1 (Connect-PnPOnline -AccessToken): $($_.Exception.Message)"
      }

what does not work

- task: AzurePowerShell@5
  name: ConnectPnpOnline
  inputs:
    azureSubscription: DEV_Connection
    azurePowerShellVersion: latestVersion
    ScriptType: InlineScript
    Inline: |
      $url = "https://$(tenantName).sharepoint.com"
      Write-Host "##[debug]Connecting to $url/sites/$(siteName)"

      Write-Host "##[group]Install/Import  PS modules"
      Install-Module PnP.PowerShell -Scope "CurrentUser" -Verbose -AllowClobber -Force
      Write-Host "##[endgroup]"

      try {
        $conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -ManagedIdentity  -ReturnConnection
        Write-Host "##[debug]Get-PnPConnection"
        Write-Host $conn.Url
      }
      catch {
          Write-Host "##[error] 1 (Connect-PnPOnline -AccessToken): $($_.Exception.Message)"
      }

In the second case, when using -ManagedIdentity, I get the following error: {"error":"invalid_request","error_description":"Identity not found"}

[gautamdsheth]

Hello, we have implemented the Azure AD Workload identity based on this: https://azure.github.io/azure-workload-identity/docs/installation.html

Is it different compared to what you are using ? Also, do you have any reference to sample code on how we can get the access token ? Totally open to adding it here.

We have implemented it based on sample code available here:

https://github.com/Azure/azure-workload-identity/blob/main/examples/msal-net/akvdotnet/TokenCredential.cs

[kkazala]

@gautamdsheth Please have a look at my post at dev.to, where I described what I'm doing step by step.

Bottomline: to sign in to SPO site I need to execute the following:

 $azAccessToken = Get-AzAccessToken -ResourceUrl $url
 $conn = Connect-PnPOnline -Url "$url/sites/$(siteName)" -AccessToken $azAccessToken.Token -ReturnConnection

# and test using the returned connection as a parameter
 Add-PnPListItem -List "test" -Values @{"Title"="$(Build.BuildId)"} -Connection $conn       

Executing the Connect-PnPOnline -Url $url -AzureADWorkloadIdentity command gives me an error

[JakeStanger]

Confusingly, Workload Identity Federation seems to be different to Workload Identity - Federation is the new service principal auth method using OIDC, whereas WI just seems to be Kubernetes specific.

Would be very happy to be corrected on this, but the docs and testing seems to indicate that SPO does not support Workload Indentity Federation. It only supports certificate auth, and fails with HTTP 401 for tokens obtained with other auth types. This would make support in PnP PowerShell unfortunately rather limited.

[kkazala]

@JakeStanger thx for looking into it. Yes, I noticed that the pipeline with Workload Identity Federation is creating a service principal. I wasn't sure if AzureADWorkloadIdentity would be the parameter to use in this case. I sometimes try things to find out =) When using the Workload Identity Federation, two resources are created: Service Principal and App Registration. This is good news, because we can now grant the App Registration API permissions (I'm using Sites.Selected for Graph and SPO APIs), and grant access to a specific SPO site

"It only supports certificate auth, and fails with HTTP 401 for tokens obtained with other auth types." One thing I'm sure is that my approach, as described above, works. I just executed the pipeline again and it still works today. I think the key here is that I'm not really authenticating. I'm just refreshing the token 🙃Using refresh_token to exchange Azure/Graph token for a SPO token is not new anyway

So I guess the question is- where does it fit into the authentication model? It would be great if we could have it embedded in the pnp Powershell, instead of executing the above two lines separately.

Can you conform that the example I included above works for you? And if yes, do you think we could have it supported natively by PnP? Maybe another parameter?

[JakeStanger]

To clarify:

"It only supports certificate auth, and fails with HTTP 401 for tokens obtained with other auth types."

This was only referring to the other service principal auth types, so client secret & WIF.

In your linked approach, you rely on the AzurePowerShell step which you pass the ARM connection into. Internally this must be obtaining the WIF token, which Get-AzAccessToken gets. I think you're then relying on an interesting quirk that enterprise apps can act as managed identities? That's a clever workaround, since managed identity auth to SPO works fine.

With that extra knowledge, I'm on board with what you're saying. It would be ideal if DevOps and PnP.PowerShell integrated smoothly with the Managed Identity. I do wonder how feasibile that is to implement though?

It would also be ideal, to provide a slightly less hacky approach in general support a wider range of scenarios, if the SPO guys gave us the ability to use WIF directly through a service principal, without any of the enterprise app faff. I appreciate that's out of scope for this though :)

---

I'll give your blog post a go and see if that gives me something to work with though, that's very useful thanks.

[kkazala]

@JakeStanger lol, didn't notice the lack of "collaborator" tag next to your name and assumed you are part of the PnP team 🙈 Sorry about that 🤷🏻‍♀️

[JakeStanger]

No worries! I've just given it a go and it looks like it may actually be a lot simpler than we thought.

It looks like you don't need to worry about any of the enterprise app or managed identity stuff - I just granted permissions onto the normal old app registration through the portal, set up the WIF/ARM connection (as you'd expect, just copying values back/forth with the web interface) and it Just Werked:tm: I didn't even need to look at the Enterprise Application or run any scripts (other than the actual pipeline one).

You still need the workaround to use Get-AzAccessToken and pass that to Connect-PnPOnline, so there's definitely room for improvement there, but that massively simplifies the setup.

[gautamdsheth]

@JakeStanger / @kkazala - Ahh Azure and naming 😂

So, what we implemented was something kubernetes specific. What this feature request is about is Workload Identity Federation.

@JakeStanger - can we reverse engineer/check how Az PowerShell obtains the tokens in WIF scenario ? Would be happy to include that here ? Also, this parameter, -WorkloadIdentity, since it is confusing, should we rename it or something ? We can do it since we haven't GA'ed this , would love to hear your thoughts on this.

[kkazala]

@JakeStanger "It looks like you don't need to worry about any of the enterprise app or managed identity stuff " about that... I actually want to "worry about it". Managed Identity is probably my favorite capability in Azure. It improves security of the solution and allows me to stop worrying about all of the app registration secrets/certificates stuff. ;) So even though you can use my "hack" in a service connection configured with a service principal, this is not what my ticket is about (as @gautamdsheth correctly pointed out 👍)

Do you think we should have yet another parameter, clearly pointing out it's AzureDevOps?

[JakeStanger]

Also, this parameter, -WorkloadIdentity, since it is confusing, should we rename it or something ?

The more I look into this, the more confusing it gets. From this page, I think what it's saying is that the whole thing is called Workload Identity Federation, and Workload Identity (the Kubernetes thing) is a supported scenario of that.

If it remains specific, you could rename it to AksWorkflowIdentity so it's clear it belongs to Azure Kubernetes Service. AKS definitely would need to be mentioned in any help/description text at least.

It may actually work out, depending on implementation details, that you can support all scenarios in which case changing it to WorkloadIdentityFederation would probably make the most sense.

It improves security of the solution and allows me to stop worrying about all of the app registration secrets/certificates stuff. ;)

@kkazala Using WIF effectively accomplishes the same thing - you can use it in place of secrets/certificates and don't need any credentials to perform the auth :) It arguably makes more sense in this context too, since Managed Identities are for resources within your Azure tenant, which the DevOps hosted agents technically are not.

---

can we reverse engineer/check how Az PowerShell obtains the tokens in WIF scenario ?

I can't see why not. So there's sorta two big steps in the process:

1. The AzurePowerShell@5 DevOps step has the code responsible for creating the auth. It gets details from context and uses those to fetch a federated token, then passes that to Azure PowerShell in order to auth it. You can see in the logs this actually runs:

   Connect-AzAccount -ServicePrincipal -Tenant *** -ApplicationId *** -FederatedToken ***** -Environment AzureCloud -Scope Process

2. More relevant to us - It looks like Get-AzAccessToken eventually reaches out to the Azure.Identity credential helpers, requesting a token. It also rather helpfully caches tokens, so repeat calls returns the same token.

It may actually be relatively simple, and all that is required is to use the AzurePowerShellCredential helper to get the cached token.

If not there is also a WorkloadIdentityCredential class. That'll require a bit more reverse-engineering etc to get the context out from the current environment, and I think it might be Kubernetes-specific, but again not very clear.

There are plenty of other credential providers and I wouldn't be surprised if one of those does the job, like the cache ones or something.

Otherwise, PnP PowerShell could probably just support the auth flow itself. I think there's a very specific set of values you need to put in specific places along the way to make SPO auth work (Graph is easy...) but it's clearly possible. Getting a federated OIDC token out of DevOps would likely have to be a manual step, but that could then be passed to get a valid auth token:

https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-client-creds-grant-flow#third-case-access-token-request-with-a-federated-credential

[kkazala]

@gautamdsheth how about we make it simple? The Connect command could simply use existing token and Connect-PnPOnline -Url $url -CurrentAccessToken would simply execute rhe code I pasted above

My idea is that although this approach is nothing new, I think many people still authenticate using service principal name and certificate. Adding CurrentAccessToken parameter would make it easier to use this approach

[eduardpaul]

Confusingly, Workload Identity Federation seems to be different to Workload Identity - Federation is the new service principal auth method using OIDC, whereas WI just seems to be Kubernetes specific.

Would be very happy to be corrected on this, but the docs and testing seems to indicate that SPO does not support Workload Indentity Federation. It only supports certificate auth, and fails with HTTP 401 for tokens obtained with other auth types. This would make support in PnP PowerShell unfortunately rather limited.

@JakeStanger thanks for the link to docs, it was really hard to understand why I could connect and read web props using the access token but, as soon as i try to install spfx solution I get the "The remote server returned an error: (401) Unauthorized.".

and @kkazala thanks for the post explaining all this! I really think WIF should be the way to go for teams working with DevOps+M365. Certificate/password rotation is a must (and a mess) :)

[kkazala]

@gautamdsheth Is there any progress here? Do you think that simply refreshing the token, like I did, might be an option?

I wonder if it would make sense to open a ticket with Azure Identity team, to request including the new Workload Identity Federation (DevOps) in the supported credential types and then PnP could use it to give us correct context? But it would already speed the development up a lot if we could use Connect-PnPOnline -Url $url -CurrentAccessToken when executing the code is a context of already authenticated user (be it Azure Functions, Pipeline or local dev), with pnp simply refreshing the tokens

[gautamdsheth]

@kkazala - can you try with Connect-PnPOonline -AzureADWorkloadIdentity and let us know ?

[kkazala]

@gautamdsheth I tried it before, and according to the documentation the AzureADWorkloadIdentity is for another case. Should I try again?

Am I correct that when signing in, you are using the same approach as Azure SDK for .NET? There's a ticket AzurePipelinesCredential parameterless constructor and add to DAC about adding support for the Workload Identity Federation to the DefaultAzureCredential

Perhaps once it's done, you could simple use the same approach or, even better, add a -DefaultAzureCredential switch? =) What do you think?