Open sdfarquhar opened 10 months ago
I have been doing some additional digging and decided to try and run the test connection commands int he MSCloudLoginAssitant module since this is what is called on from the MS365 DSC modules to make the initial connection.
I tried both commands below for workloads MS Graph and EXO using my Azure Automation managed identity and got connection failures. (I used both the current 1.0.121 version as well as the 1.0.120 version of MSCloudLoginAssistant)
Test-MSCloudLogin -Identity -Platform MicrosoftGraph -TenantID mytenant.onmicrosoft.com -Verbose Test-MSCloudLogin -Identity -Platform ExchangeOnline -TenantID mytenant.onmicrosoft.com -Verbose
Since the 1.0.121 version was working for me last week I am starting to suspect that something may have changed in the way Microsoft is handling managed identities within Azure Automation. Is there a way for someone to verify this or is there additional troubleshooting I can do to get to a root cause?
Could you try this example within your runbook:
Thanks for the link. I'm not sure what part I'm supposed to look at but I ran this set of commands and got the output below.
Disable-AzContextAutosave -Scope Process
$AzureContext = (Connect-AzAccount -Identity).context $AzureContext
$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext $AzureContext
Mode : Process ContextDirectory : ContextFile : CacheDirectory : CacheFile : KeyStoreFile : Settings : {}
Name : Account : MSI@50342 Environment : AzureCloud Subscription : HIDDEN FOR DATA CONCERNS Tenant : HIDDEN FOR DATA CONCERNS TokenCache : VersionProfile : ExtendedProperties : {} Name : Default Account : MSI@50342 Environment : AzureCloud Subscription : HIDDEN FOR DATA CONCERNS Tenant : HIDDEN FOR DATA CONCERNS TokenCache : VersionProfile : ExtendedProperties : {}
I also ran the commands referenced in the article to get the access token via HTTP Get and it provided my token:
$resource= "?resource=https://management.azure.com/" $url = $env:IDENTITY_ENDPOINT + $resource $Headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $Headers.Add("X-IDENTITY-HEADER", $env:IDENTITY_HEADER) $Headers.Add("Metadata", "True") $accessToken = Invoke-RestMethod -Uri $url -Method 'GET' -Headers $Headers Write-Output $accessToken.access_token
Ok I did some more testing with the Test-MSCloudlogin command and even tried creating a new Az Automation account to see if something was corrupted in the managed identify. I found that it appears the command fails when I try to use the ExchangeOnline platform but works with AzureAD. Something must have changed recently with the way these workloads allow connections from system managed identities?
Could you try this example within your runbook:
Hello, I was wondering if there was any new information based on what I had tested? I also tested running the "Test-MSCloudlogin" commands in a 7.2 version of PowerShell with the same connection timeout for workloads other than AzureAD. Is it possible MS made a recent change to Azure Automation that is preventing connections using a system managed identity?
UPDATE: I think this might be related to the 2.9 version of the Microsoft.Graph commands. There is a current known bug (https://github.com/microsoftgraph/msgraph-sdk-powershell/issues/2424) that is preventing people from connecting. I tried running connect-mggraph -Identity with the 2.9 version and got a similar connection failure as others. I then downgraded the Microsoft.Graph.Authentication cmdlet to 2.8 and was able to run the connect-mggraph command successfully. I then retried the test-mscloudlogin command however it still times out. I suspect that the the test-mscloudlogin probably is dependent on several of the Microsoft.Graph.* cmdlets.
See the error below I got running the export command using debug. Its indicating that it can't write to the event log and is failing to get a user response (I am running this in Azure Automation using a system managed identity so there is no ability to provide a user response). This was working 3 weeks ago and then suddenly stopped work with these connection errors. Updating versions of Microsoft365DSC, MS Graph, etc. have not fixed this. Could MS have changed the way the Azure Automation managed identities are working?
*Command I ran: Export-M365DSCConfiguration -ManagedIdentity -Mode Full -Components @("EXOTransportRule") -TenantID mytenant.onmicrosoft.com -path $Cpath >&1 -Verbose -Debug**
PowerShell meta provider initialization failed.
Exception calling "SourceExists" with "1" argument(s): "The source was not found, but some or all event logs could not be searched. Inaccessible logs: Security, State."
Could not write to event log Source {M365DSCReverse::Test-M365DSCModuleValidity} EntryType {Information} Message {Exception calling "ShouldContinue" with "2" argument(s): "A command that prompts the user failed because the host program or the command type does not support user interaction. The host was attempting to request confirmation with the following message: PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or 'C:\Users\ContainerUser\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the NuGet provider by running 'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want PowerShellGet to install and import the NuGet provider now?"} { Exception calling "SourceExists" with "1" argument(s): "The source was not found, but some or all event logs could not be searched. Inaccessible logs: Security, State." } \ at Add-M365DSCEvent, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCLogEngine.psm1: line 194 \ at Start-M365DSCConfigurationExtract, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCReverse.psm1: line 102 \ at Export-M365DSCConfiguration, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCUtil.psm1: line 1320 \ at
{ Exception calling "SourceExists" with "1" argument(s): "The source was not found, but some or all event logs could not be searched. Inaccessible logs: Security, State." } \ at Add-M365DSCEvent, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCLogEngine.psm1: line 194 \ at Start-M365DSCConfigurationExtract, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCReverse.psm1: line 102 \ at Export-M365DSCConfiguration, C:\usr\src\PSModules\Microsoft365DSC\modules\M365DSCUtil.psm1: line 1320 \ at
So I was able to get past this issue by first doing a normal connect to Exchange online before running the extract.
Connect-ExchangeOnline -ManagedIdentity -Organization mytenant.onmicrosoft.com
Export-M365DSCConfiguration -ManagedIdentity -Mode Full -Workloads @("EXO") -TenantID mytenant.onmicrosoft.com -path $path *>&1 | out-null
I believe the issues is linked to the MSCloudLoginAssitant 1.1.0, specifically the function, Connect-M365Tenant and one of the ElseIf statements, see code snippet from there below. I believe what is happening is since I already am connected to the EXO workload its bypassing some part of this section that is causing the timeout.
if ($null -eq $Global:MSCloudLoginConnectionProfile) { $Global:MSCloudLoginConnectionProfile = New-Object MSCloudLoginConnectionProfile }
elseif ( $Global:MSCloudLoginConnectionProfile.$workloadInternalName.Connected `
-and (Compare-InputParametersForChange -CurrentParamSet $PSBoundParameters))
{
Write-Verbose -Message 'Resetting connection profile'
$Global:MSCloudLoginConnectionProfile.$workloadInternalName.Connected = $false
}
Is there any update if this is going to be fixed? I can work around the SPO and EXO workloads by first running the connect- commands to connect to EXO or SPO however all other workloads do not work.
Description of the issue
I am running the latest M365 DSC in an Azure Automation to run an export using a managed ID. This worked early last week but appeared to break after updating to 1.23.1101.1 with the error below. I made sure all support Graph modules were updated to 2.8.0 but this has not fixed the issue. Any ideas what might be going on?
Failed Unable to connect to the remote server (Unable to connect to the remote server (A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond XXX.XX.XX.XXX:80)
Microsoft 365 DSC Version
1.23.1101.1
Which workloads are affected
Planner
The DSC configuration
Verbose logs showing the problem
Environment Information + PowerShell Version
No response