Account Connection failure on self-hosted agent when using AzurePowerShell@4, AzurePowerShell@5 with Sovereign Environment via Managed Identity

cutecycle commented 3 years ago

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature? Bug Type: Bug

Enter Task Name: AzurePowerShell

list here (V# not needed): 4,5 https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

Server - Azure Pipelines or TFS on-premises?
- If using TFS on-premises, provide the version:
- If using Azure Pipelines, provide the account name, team project name, build definition name/build number: Microsoft, redacted, redacted, redacted (internal)
Agent - Hosted or Private:
- If using Hosted agent, provide agent queue name: Not entirely sure of policy regarding publicly sharing agent queue names
- If using private agent, provide the OS of the machine running the agent and the agent version: Windows Server 2019 Datacenter 17763.1637, VSTS Agent 2.179, AzureUSGovernment

Issue Description

We have a system in our Azure Pipelines (YAML) deployment that conditionally splits powershell tasks that reference Azure resources between AzureCloudand USGov, because we are having issues using an AzurePowerShelltask when:

1) AzureCloud has a service principal-based service connection to a microsoft-hosted agent 2) USGov has a managed identity-on-VM-agent-based service connection.

AzureResourceManagerTemplateDeployment@3
AzureFileCopy@4
SqlAzureDacpacDeployment@1

work with the USGov service connection,

but theAzurePowerShell@5 task doesn't.

It does seem to correctly realize that the Service Connection is a USGov one based on $endpointObject.environment:

2020-12-14T17:43:21.4000793Z ##[debug]INPUT_CONNECTEDSERVICENAMEARM: '***'
2020-12-14T17:43:21.4004640Z ##[debug]ENDPOINT_URL_***: 'https://management.usgovcloudapi.net/'
2020-12-14T17:43:21.4031732Z ##[debug]ENDPOINT_AUTH_***: '********'
2020-12-14T17:43:21.4107779Z ##[debug]ENDPOINT_DATA_***: '{"environment":"AzureUSGovernment","scopeLevel":"Subscription","subscriptionId":"******","subscriptionName":"*****","environmentUrl":"https://management.usgovcloudapi.net/","galleryUrl":"https://gallery.usgovcloudapi.net/","serviceManagementUrl":"https://management.core.usgovcloudapi.net/","resourceManagerUrl":"https://management.usgovcloudapi.net/","activeDirectoryAuthority":"https://login.microsoftonline.us/","environmentAuthorityUrl":"https://login.microsoftonline.us/","graphUrl":"https://graph.windows.net/","managementPortalUrl":"https://manage.windowsazure.us/","armManagementPortalUrl":"https://portal.azure.us","activeDirectoryServiceEndpointResourceId":"https://management.core.usgovcloudapi.net/","sqlDatabaseDnsSuffix":".database.usgovcloudapi.net","AzureKeyVaultDnsSuffix":"vault.usgovcloudapi.net","AzureKeyVaultServiceEndpointResourceId":"https://vault.usgovcloudapi.net","StorageEndpointSuffix":"core.usgovcloudapi.net","EnableAdfsAuthentication":"false"}'

But eventually fails to login:

020-12-14T17:43:25.4443556Z ##[command]Connect-AzAccount -Identity @processScope
2020-12-14T17:43:26.8625685Z ##[command] Set-AzContext -SubscriptionId *** -TenantId ***
2020-12-14T17:43:26.9393026Z ##[debug]Leaving Initialize-AzModule.
2020-12-14T17:43:26.9623941Z ##[error]Initializing Az module failed: For troubleshooting, refer: https://aka.ms/azurepowershelltroubleshooting
2020-12-14T17:43:26.9632364Z ##[debug]Processed: ##vso[task.logissue type=error]Initializing Az module failed: For troubleshooting, refer: https://aka.ms/azurepowershelltroubleshooting
2020-12-14T17:43:26.9633671Z ##[debug]Caught exception from task script.
2020-12-14T17:43:26.9641810Z ##[debug]Error record:
2020-12-14T17:43:27.0517810Z ##[debug]Set-AzContext : Please provide a valid tenant or a valid subscription.
2020-12-14T17:43:27.0530114Z ##[debug]At C:\agent\_work\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\4.176.3\ps_modules\VstsAzureHelpers_\InitializeAzModuleFunctions.ps1:190 char:13
2020-12-14T17:43:27.0543734Z ##[debug]+     $null = Set-AzContext -SubscriptionId $SubscriptionId @additional
2020-12-14T17:43:27.0587035Z ##[debug]+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-14T17:43:27.0614976Z ##[debug]    + CategoryInfo          : CloseError: (:) [Set-AzContext], ArgumentException
2020-12-14T17:43:27.0631319Z ##[debug]    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Profile.SetAzureRMContextCommand
2020-12-14T17:43:27.0651161Z ##[debug]

As a result, we and others in related teams are using conditional branches and split stages where an environment using AzureUSGovernmentexecutes a similar task, but with PowerShell@2 with Connect-AzAccount:

AzureCloud:

            - task: AzurePowerShell@5
              displayName: "Enable data factory triggers (AzureCloud, Service Principal, AzurePowerShell)"
              condition: eq('${{parameters.cloudEnvironment}}', 'AzureCloud')
              inputs:
                azurePowerShellVersion: latestVersion
                azureSubscription: "${{ parameters.resourceManagerConnection }}"
                scriptType: filePath
                scriptPath: $(Pipeline.Workspace)/${{ parameters.artifactName }}/ArmDeployers/TriggerManager.ps1
                scriptArguments: ...arguments...

AzureUSGovernment:

            - task: PowerShell@2
              displayName: "Enable data factory triggers (Sovereign, VM Managed Identity, PowerShell)"
              condition: ne('${{parameters.cloudEnvironment}}', 'AzureCloud')
              inputs:
                targetType: inline
                script: |
                  Connect-AzAccount -Identity -Environment ${{ parameters.cloudEnvironment }} -Subscription "${{ parameters.subscriptionId}}" -Tenant ${{ parameters.tenantId}}

                  ...script...

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

2020-12-14T17:43:26.9632364Z ##[debug]Processed: ##vso[task.logissue type=error]Initializing Az module failed: For troubleshooting, refer: https://aka.ms/azurepowershelltroubleshooting
2020-12-14T17:43:26.9633671Z ##[debug]Caught exception from task script.
2020-12-14T17:43:26.9641810Z ##[debug]Error record:
2020-12-14T17:43:27.0517810Z ##[debug]Set-AzContext : Please provide a valid tenant or a valid subscription.
2020-12-14T17:43:27.0530114Z ##[debug]At C:\agent\_work\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\4.176.3\ps_modules\VstsAzureHelpers_\InitializeAzModuleFunctions.ps1:190 char:13

cutecycle commented 3 years ago

Additional note:

reference: it may be that the task does not branch based on managed identity?

https://github.com/microsoft/azure-pipelines-tasks/blob/c8f0311b276522b1cbbc202298b1e05796128f13/Tasks%2FAzurePowerShellV5%2FInitializeAz.ps1#L79

AmrutaKawade commented 3 years ago

@N-Usha can you check this enhancement request?

cutecycle commented 3 years ago

I've opened a PR to fix this: https://github.com/microsoft/azure-pipelines-tasks/pull/14533

AmrutaKawade commented 3 years ago

These changes are already merged closing this issue

microsoft / azure-pipelines-tasks