Open AErmie opened 4 days ago
Hello @AErmie , thanks for your feedback. Action group settings like Emails Logic App, Azure Function, Arm Roles, Event Hubs, Webhooks are not part of the policy compliance since it is not really possible to control the entire list. Should you need to update, you need to:
There's absolutely no need to remove the entire amba-alz deployment.
Thanks, Bruno.
Thanks for that confirmation @Brunoga-MS, however, when I attempt to run the Start-AMBA-ALZ-Maintenance.ps1
script (with a targeted -cleanItems NotificationAssets
), it doesn't seem to actually do anything. All it does is return the number of Management Groups that are childs of the $pseudoRootManagementGroup
.
Am I executing it incorrectly?
Output
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-ExecutionPolicy
Unrestricted
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> $pseudoRootManagementGroup = "bcgov-managed-lz-forge"
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> ./Start-AMBA-ALZ-Maintenance.ps1 -pseudoRootManagementGroup $pseudoRootManagementGroup -cleanItems "NotificationAssets"
Found '13' management group(s) (including the parent one) which are part of the 'bcgov-managed-lz-forge' management group hierarchy, to be queried for AMBA-ALZ resources.
CleanItems Options
For anyone else that might encounter this question, and wonder what the -cleanItems
options are, here is the output from running Get-help ./Start-AMBA-ALZ-Maintenance.ps1
.
-cleanItems <String>
Required. The item type we want the script to clean up. The options are:
- Amba-Alz
- Deployments
- NotificationAssets
- Alerts
- PolicyAssignments
- PolicyDefinitions
- OrphanedAlerts
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.Resources
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 7.7.0 Az.Resources Core,Desk {Export-AzResourceGroup, Export-AzTemplateSpec, Get-AzDenyAssignment, Get-AzDeployment…}
Script 6.11.1 Az.Resources Core,Desk {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment…}
Script 3.2.0 Az.Resources Core,Desk {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ResourceGraph
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.0.1 Az.ResourceGraph Core,Desk {Search-AzGraph, Get-AzResourceGraphQuery, New-AzResourceGraphQuery, Remove-AzResourceGraphQuery…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ManagedServiceIdentity
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.2.1 Az.ManagedServiceIdentity Core,Desk {Get-AzFederatedIdentityCredential, Get-AzSystemAssignedIdentity, Get-AzUserAssignedIdentity, Get-AzUserAssignedIdent…
Script 1.1.1 Az.ManagedServiceIdentity Core,Desk {Get-AzFederatedIdentityCredentials, Get-AzSystemAssignedIdentity, Get-AzUserAssignedIdentity, Get-AzUserAssignedIden…
Looking at the Start-AMBA-ALZ-Maintenance.ps1 more closely, I see the first command called within the NotificationAssets
switch is Get-ALZ-ActionGroups
.
Switch ($cleanItems)
{
"NotificationAssets"
{
# Invoking function to retrieve action groups
$agToBeDeleted = Get-ALZ-ActionGroups
...
When I look at that Function in the PowerShell script, we see this:
Function Get-ALZ-ActionGroups {
# get action groups to delete
$query = "resources | where type =~ 'Microsoft.Insights/actionGroups' | where tags['_deployed_by_amba'] =~ 'True' | project id"
$actionGroupIds = Search-AzGraphRecursive -Query $query -ManagementGroupNames $managementGroups | Select-Object -ExpandProperty Id | Sort-Object | Get-Unique
Write-Host "- Found '$($actionGroupIds.Count)' action group(s) with tag '_deployed_by_amba=True' to be deleted." -ForegroundColor Cyan
# Returning items
$actionGroupIds
}
When I attempt to manually run each command individually, the $query
command completes successfully, whereas the $actionGroupIds
command throws this error:
PS > $query = "resources | where type =~ 'Microsoft.Insights/actionGroups' | where tags['_deployed_by_amba'] =~ 'True' | project id"
PS > $actionGroupIds = Search-AzGraphRecursive -Query $query -ManagementGroupNames $managementGroups | Select-Object -ExpandProperty Id | Sort-Object | Get-Unique
Search-AzGraphRecursive: The term 'Search-AzGraphRecursive' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
It seems to not like the Recursive
part, and the Search-AzGraph doesn't even have a Recursive
parameter.
But even if I change it to $actionGroupIds = Search-AzGraph -Recursive -Query $query -ManagementGroupNames $managementGroups | Select-Object -ExpandProperty Id | Sort-Object | Get-Unique
it throws the error: Search-AzGraph: A parameter cannot be found that matches parameter name 'Recursive'.
If I remove Recursive
completed, to $actionGroupIds = Search-AzGraph -Query $query -ManagementGroupNames $managementGroups | Select-Object -ExpandProperty Id | Sort-Object | Get-Unique
it throws the error: Search-AzGraph: A parameter cannot be found that matches parameter name 'ManagementGroupNames'.
If I execute $actionGroupIds = Search-AzGraph -Query $query
, and then review the results, I see:
Sample output
id Re
so
ur
ce
Id
-- --
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-bcgov-mgd-lz-forge-connectivity-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-bcgov-mgd-lz-forge-connectivity-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-abc123-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-abc123-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-bcgov-mgd-lz-forge-management-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-bcgov-mgd-lz-forge-management-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-fbab12-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-fbab12-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-a1fb6f-test-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-a1fb6f-test-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-e833c2-tools-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-e833c2-tools-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-b3fc30-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-b3fc30-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-e833c2-dev-001 /…
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/bcgov-managed-lz-forge-amba-monitoring/providers/Microsoft.Insights/actiongroups/ag-AMBA-SH-e833c2-dev-001 /…
If I remove the Recursive
and -ManagementGroupNames $managementGroups
portions of the command, using $actionGroupIds = Search-AzGraph -Query $query | Select-Object -ExpandProperty Id | Sort-Object | Get-Unique
it throws the error: Select-Object: Property "Id" cannot be found.
Even though the output from
$actionGroupIds
shows ID as lowercase, the same error is experienced regardless of case.
However, if I remove the -ExpandProperty Id
portion of the command as well, using $actionGroupIds = Search-AzGraph -Query $query | Select-Object | Sort-Object | Get-Unique
, that command completes without error. Then the Write-Host ($actionGroupIds.Count)
command has the proper count.
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Write-Host "- Found '$($actionGroupIds.Count)' action group(s) with tag '_deployed_by_amba=True' to be deleted." -ForegroundColor Cyan
- Found '16' action group(s) with tag '_deployed_by_amba=True' to be deleted.
At this point, I don't know if the $actionGroupIds
object is in the right/expected format for use with the rest of the script.
Hi @AErmie , looking at the syntax you shared it seems the command was correctly entered. I have a couple of questions:
That aside, you should expect an output similar to the one below (only reporting the last two items: alert processing rules an action groups)
Let me know if (and how) I can reach out to you for a troubleshooting session.
Thanks, Bruno.
Hi @Brunoga-MS, thanks for confirming the command. Everything seems to be correct. I even downloaded a fresh clone of the AMBA repo this morning, but encountered the same results.
The Management Group value doesn't seem to be the issue, since the script does correctly identify 13 other Management Groups that are a part of the hierarch.
Here's the details from all the commands I ran.
$PSVersionTable
Name Value
---- -----
PSVersion 7.4.6
PSEdition Core
GitCommitId 7.4.6
OS Microsoft Windows 10.0.22631
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
...
$pseudoRootManagementGroup = "bcgov-managed-lz-forge"
$pseudoRootManagementGroup
bcgov-managed-lz-forge
...
Connect-AzAccount
Please select the account you want to login with.
Retrieving subscriptions for the selection...
Subscription name Tenant
----------------- ------
bcgov-mgd-lz-forge-management Government of BC
...
Get-ExecutionPolicy
Unrestricted
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> ./Start-AMBA-ALZ-Maintenance.ps1 -pseudoRootManagementGroup $pseudoRootManagementGroup -cleanItems NotificationAssets -WhatIf -Verbose
VERBOSE: Populating RepositorySourceLocation property for module Az.Resources.
VERBOSE: Populating RepositorySourceLocation property for module Az.Resources.
VERBOSE: Populating RepositorySourceLocation property for module Az.Resources.
VERBOSE: Populating RepositorySourceLocation property for module Az.ResourceGraph.
VERBOSE: Populating RepositorySourceLocation property for module Az.ManagedServiceIdentity.
VERBOSE: Populating RepositorySourceLocation property for module Az.ManagedServiceIdentity.
VERBOSE: Loading module from path 'C:\Users\adine\Documents\PowerShell\Modules\Az.ManagedServiceIdentity\1.1.1\Az.ManagedServiceIdentity.psm1'.
Found '13' management group(s) (including the parent one) which are part of the 'bcgov-managed-lz-forge' management group hierarchy, to be queried for AMBA-ALZ resources.
Would having multiple versions of the required PowerShell modules cause any issues?
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.Resources
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 7.7.0 Az.Resources Core,Desk {Export-AzResourceGroup, Export-AzTemplateSpec, Get-AzDenyAssignment, Get-AzDeployment…}
Script 6.11.1 Az.Resources Core,Desk {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment…}
Script 3.2.0 Az.Resources Core,Desk {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ResourceGraph
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.0.1 Az.ResourceGraph Core,Desk {Search-AzGraph, Get-AzResourceGraphQuery, New-AzResourceGraphQuery, Remove-AzResourceGraphQuery…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ManagedServiceIdentity
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.2.1 Az.ManagedServiceIdentity Core,Desk {Get-AzFederatedIdentityCredential, Get-AzSystemAssignedIdentity, Get-AzUserAssignedIdentity, Get-AzUserAssignedIdent…
Script 1.1.1 Az.ManagedServiceIdentity Core,Desk {Get-AzFederatedIdentityCredentials, Get-AzSystemAssignedIdentity, Get-AzUserAssignedIdentity, Get-AzUserAssignedIden…
If you have time to connect, that would be great! Please email me at: Adin.Ermie@outlook.com and Adin.Ermie@gov.bc.ca.
Here's a link to the Gist, where I captured all the -Debug
output from the command-line: https://gist.github.com/AErmie/e74265587f17245186cb9e687fa97b18
Interestingly, in this debug output, it actually shows the list of Management Groups, Action Groups (16 in total).
I had another thought that I tested...
Does it matter which account originally deployed the AMBA policy initiatives? We're using a GitHub Workflow to do so. And so I created an alternative workflow to execute ./Start-AMBA-ALZ-Maintenance.ps1 -pseudoRootManagementGroup $pseudoRootManagementGroup -cleanItems NotificationAssets -WhatIf
.
What's interesting is if we compare what I have in theGitHub workflow versus what the output actually is, in the command-line I'm passing -cleanItems NotificationAssets
whereas the actual execution shows ... to be queried for AMBA-ALZ resources
.
❓ Does that mean it's somehow defaulting to the full "Amba-Alz" option? But even if that were so, you'd think the output would show everything (like in your screenshot @Brunoga-MS ).
GitHub Workflow (partial)
- name: Remove Notification Assets
id: deploy_amba
shell: pwsh
run: |
cd ./azure_monitor_baseline_alerts
$pseudoRootManagementGroup = "bcgov-managed-lz-forge"
./Start-AMBA-ALZ-Maintenance.ps1 -pseudoRootManagementGroup $pseudoRootManagementGroup -cleanItems NotificationAssets -WhatIf
GitHub Action Output (partial)
##[debug]Evaluating condition for step: 'Remove Notification Assets'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Remove Notification Assets
##[debug]Loading inputs
##[debug]Loading env
Run cd ./azure_monitor_baseline_alerts
##[debug]/usr/bin/pwsh -command ". '/home/runner/work/_temp/6d914d47-38f4-464a-ae8d-1a9[2](https://github.com/BC-Gov-PaaS-Platform-Services/azure-lz-core-forge/actions/runs/12075526335/job/33675523535#step:5:2)03286e97.ps1'"
Found '13' management group(s) (including the parent one) which are part of the 'bcgov-managed-lz-forge' management group hierarchy, to be queried for AMBA-ALZ resources.
Error: Process completed with exit code 1.
##[debug]Finishing: Remove Notification Assets
It can't be a permissions issue, as I've been able to execute the remediation process from the same PowerShell prompt (authenticated with the same account).
@AErmie as long as the account you are using now has the required permissions, it does not matter if it is different. The message you see calling Amba-ALZ is because we are referring to the ALZ pattern of AMBA. None of the parameters have default values, since we do want customer make their choice. and it is just by coincidence that it matches one of the parameter' value. As far as the output goes, what is shown depends on the cleanItems param choice. For notifications it only show the following items:
I suspect something with multiple module version. In may case I am running the following versions and only have 1 module per version:
Could you please try to remove older version?
Thanks for confirming @Brunoga-MS that it doesn't matter which account originally applied the AMBA.
I have removed all old versions of the modules, but still experience the same issue.
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.Accounts
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 4.0.0 Az.Accounts Core,Desk {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.Resources
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 7.7.0 Az.Resources Core,Desk {Export-AzResourceGroup, Export-AzTemplateSpec, Get-AzDenyAssignment, Get-AzDeployment…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ResourceGraph
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.0.1 Az.ResourceGraph Core,Desk {Search-AzGraph, Get-AzResourceGraphQuery, New-AzResourceGraphQuery, Remove-AzResourceGraphQuery…}
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-Module -ListAvailable Az.ManagedServiceIdentity
Directory: C:\Users\adine\Documents\PowerShell\Modules
ModuleType Version PreRelease Name PSEdition ExportedCommands
---------- ------- ---------- ---- --------- ----------------
Script 1.2.1 Az.ManagedServiceIdentity Core,Desk {Get-AzFederatedIdentityCredential, Get-AzSystemAssignedIdentity, Get-AzUserAssignedIdentity, Get-AzUserAssignedIdent…
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> $pseudoRootManagementGroup = "bcgov-managed-lz-forge"
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Connect-AzAccount
Please select the account you want to login with.
Retrieving subscriptions for the selection...
Subscription name Tenant
----------------- ------
bcgov-mgd-lz-forge-management Government of BC
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> Get-ExecutionPolicy
Unrestricted
PS C:\Users\adine\Documents\GitHub\BC-Gov-Repos\azure-monitor-baseline-alerts\patterns\alz\scripts> ./Start-AMBA-ALZ-Maintenance.ps1 -pseudoRootManagementGroup $pseudoRootManagementGroup -cleanItems NotificationAssets -WhatIf
Found '13' management group(s) (including the parent one) which are part of the 'bcgov-managed-lz-forge' management group hierarchy, to be queried for AMBA-ALZ resources.
What's also very interesting, is that I tried repeating all of these steps from a completely different system (ie. a VM running in Azure), and it behaved exactly the same way, with exactly the same output!
Check for previous/existing GitHub issues
Issue Type?
Question
Description
When we originally deployed the AMBA Landing Zone initiatives, we included an individual email within the
ALZMonitorActionGroupEmail
property.Now, when testing changes to that value, although we've modified it (ie. removed it, so
ALZMonitorActionGroupEmail
is an empty array), although the deployment completes successfully (and we executed the Remediation process), none of the Azure Groups are actually updated.Is this "by design" and expected behaviour? To make an update like this (or even to add another email in the list), do we actually have to destroy and re-deploy the entire AMBA? What is the recommended approach for making updates to the Action Groups deployed?