Closed bojanmisic closed 2 months ago
Raw pipeline log:
2023-08-02T09:56:43.5509090Z ##[section]Starting: Hub (Hub-and-Spoke) Deployment
2023-08-02T09:56:43.5513881Z ==============================================================================
2023-08-02T09:56:43.5514010Z Task : Azure PowerShell
2023-08-02T09:56:43.5514082Z Description : Run a PowerShell script within an Azure environment
2023-08-02T09:56:43.5514186Z Version : 5.225.1
2023-08-02T09:56:43.5514257Z Author : Microsoft Corporation
2023-08-02T09:56:43.5514334Z Help : https://aka.ms/azurepowershelltroubleshooting
2023-08-02T09:56:43.5514415Z ==============================================================================
2023-08-02T09:56:43.8791205Z Generating script.
2023-08-02T09:56:43.8823589Z [command]/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command . '/home/vsts/work/_temp/17a39ed3-596b-4acc-8a69-cae453b19cf8.ps1'
2023-08-02T09:56:43.8878365Z File saved!
2023-08-02T09:56:44.3602775Z ##[command]Import-Module -Name /usr/share/az_9.3.0/Az.Accounts/2.12.4/Az.Accounts.psd1 -Global
2023-08-02T09:56:49.9724358Z ##[command]Clear-AzContext -Scope Process
2023-08-02T09:56:50.0936947Z ##[command]Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue
2023-08-02T09:56:50.1451836Z ##[command]Connect-AzAccount -ServicePrincipal -Tenant REDACTED -Credential System.Management.Automation.PSCredential -Environment AzureCloud @processScope
2023-08-02T09:56:51.1152567Z
2023-08-02T09:56:51.1161750Z [32;1mName Account Subscript Environme TenantId[0m
2023-08-02T09:56:51.1162676Z [32;1m ionName nt[0m
2023-08-02T09:56:51.1163002Z [32;1m---- ------- --------- --------- --------[0m
2023-08-02T09:56:51.1166215Z Connectivity (03fcda23-1960-49c6-943d-f… abaeed0f… Connecti… AzureClo… 5432105…
2023-08-02T09:56:56.1998476Z [33;1mVERBOSE: Using Bicep v0.19.5[0m
2023-08-02T09:57:00.0634841Z [33;1mWARNING: /home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/CRML/customerUsageAttribution/cuaIdResourceGroup.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/bicepconfig.json).
2023-08-02T09:57:00.0635826Z /home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/modules/publicIp/publicIp.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/bicepconfig.json).
2023-08-02T09:57:00.0636591Z /home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/modules/hubNetworking/hubNetworking.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/modules/hubNetworking/bicepconfig.json).
2023-08-02T09:57:00.0637359Z /home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/modules/privateDnsZones/privateDnsZones.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/vsts/work/1/s/upstream-releases/v0.16.0/infra-as-code/bicep/modules/privateDnsZones/bicepconfig.json).[0m
2023-08-02T09:57:00.0819048Z [33;1mVERBOSE: Performing the operation "Creating Deployment" on target "rg-rbw-connectivity".[0m
2023-08-02T09:57:04.6953212Z [33;1mVERBOSE: 09:57:04 - Template is valid.[0m
2023-08-02T09:57:05.6148245Z [33;1mVERBOSE: 09:57:05 - Create template deployment 'alz-Hub-and-SpokeDeploy-20230802T0908508319Z'[0m
2023-08-02T09:57:05.6159102Z [33;1mVERBOSE: 09:57:05 - Checking deployment status in 5 seconds[0m
2023-08-02T09:57:11.7503640Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Network/networkSecurityGroups 'nsg-AzureBastionSubnet' provisioning status is succeeded[0m
2023-08-02T09:57:11.7504453Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-1' provisioning status is running[0m
2023-08-02T09:57:11.7505949Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Network/publicIPAddresses 'alz-ExpressRoute-Gateway-PublicIP' provisioning status is succeeded[0m
2023-08-02T09:57:11.7506955Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Resources/deployments 'pid-3f85b84c-6bad-4c42-86bf-11c233241c22-7qbbpmud4u7n4' provisioning status is running[0m
2023-08-02T09:57:11.7547888Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-0' provisioning status is running[0m
2023-08-02T09:57:11.7550186Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Network/publicIPAddresses 'alz-Vpn-Gateway-PublicIP' provisioning status is succeeded[0m
2023-08-02T09:57:11.7550666Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Resources/deployments 'pid-3f85b84c-6bad-4c42-86bf-11c233241c22-7kksjvcmpkgyy' provisioning status is running[0m
2023-08-02T09:57:11.7551107Z [33;1mVERBOSE: 09:57:11 - Resource Microsoft.Resources/deployments 'pid-2686e846-5fdc-4d4f-b533-16dcb09d6e6c-lvjmfv5dzfznm' provisioning status is running[0m
2023-08-02T09:57:11.8729121Z [33;1mVERBOSE: 09:57:11 - Checking deployment status in 12 seconds[0m
2023-08-02T09:57:24.9783425Z [33;1mVERBOSE: 09:57:24 - Resource Microsoft.Resources/deployments 'pid-3f85b84c-6bad-4c42-86bf-11c233241c22-7qbbpmud4u7n4' provisioning status is succeeded[0m
2023-08-02T09:57:24.9784471Z [33;1mVERBOSE: 09:57:24 - Resource Microsoft.Resources/deployments 'pid-3f85b84c-6bad-4c42-86bf-11c233241c22-7kksjvcmpkgyy' provisioning status is succeeded[0m
2023-08-02T09:57:24.9785423Z [33;1mVERBOSE: 09:57:24 - Resource Microsoft.Resources/deployments 'pid-2686e846-5fdc-4d4f-b533-16dcb09d6e6c-lvjmfv5dzfznm' provisioning status is succeeded[0m
2023-08-02T09:57:25.0678916Z [33;1mVERBOSE: 09:57:25 - Checking deployment status in 14 seconds[0m
2023-08-02T09:57:40.8652720Z [33;1mVERBOSE: 09:57:40 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-0' provisioning status is succeeded[0m
2023-08-02T09:57:40.8653172Z [33;1mVERBOSE: 09:57:40 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-1' provisioning status is succeeded[0m
2023-08-02T09:57:40.8653606Z [33;1mVERBOSE: 09:57:40 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-1' provisioning status is succeeded[0m
2023-08-02T09:57:40.8653990Z [33;1mVERBOSE: 09:57:40 - Resource Microsoft.Resources/deployments 'deploy-Gateway-Public-IP-0' provisioning status is succeeded[0m
2023-08-02T09:57:41.1666681Z [31;1mNew-AzResourceGroupDeployment: [0m/home/vsts/work/1/s/pipeline-scripts/Deploy-ALZHub-HubAndSpoke.ps1:30
2023-08-02T09:57:41.1667299Z [36;1mLine |
2023-08-02T09:57:41.1668141Z [36;1m 30 | [0m [36;1mNew-AzResourceGroupDeployment @inputObject[0m
2023-08-02T09:57:41.1668486Z [36;1m | [31;1m ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-08-02T09:57:41.1668820Z [31;1m[36;1m | [31;1m09:57:40 - The deployment 'alz-Hub-and-SpokeDeploy-20230802T0908508319Z'
2023-08-02T09:57:41.1669191Z [36;1m | [31;1mfailed with error(s). Showing 1 out of 1 error(s). Status Message:
2023-08-02T09:57:41.1669720Z [36;1m | [31;1mResource
2023-08-02T09:57:41.1670538Z [36;1m | [31;1m/subscriptions/03fcda23-1960-49c6-943d-f239c7de53a8/resourceGroups/rg-rbw-connectivity/providers/Microsoft.Network/ddosProtectionPlans/alz-ddos-plan referenced by resource /subscriptions/03fcda23-1960-49c6-943d-f239c7de53a8/resourceGroups/rg-rbw-connectivity/providers/Microsoft.Network/virtualNetworks/alz-hub-centralus was not found. Please make sure that the referenced resource exists. (Code: InvalidGlobalResourceReference) CorrelationId: c6905a2e-8408-4c9a-96fe-172dc2390a3a
2023-08-02T09:57:41.1671111Z [0m
2023-08-02T09:57:41.2575033Z ##[error]PowerShell exited with code '1'.
2023-08-02T09:57:41.2599725Z ##[section]Finishing: Hub (Hub-and-Spoke) Deployment
Hi @bojanmisic, thanks for calling out this issue. I was able to replicate your error when passing in false for the parDdosEnabled parameter.
After looking into it, I was able to determine that it is due to the parDdosProtectionPlanId parameter within alzDefaultPolicyAssignments.parameters.all.json being set to the DDoS protection resource ID. This value gets pre-populated automatically after using the ALZ-PowerShell-Module.
Essentially, if this parameter value is not empty then the module named modPolicyAssignmentLzsEnableDdosVnet will be deployed. This module creates a policy assignment to force all virtual networks to be linked with the DDoS plan supplied in the parDdosProtectionPlanId parameter.
To get around this, you will need to manually delete the policy assignment named "Virtual networks should be protected by Azure DDoS Protection Standard". You can then re-run the ALZ-Bicep-4a-HubSpoke workflow. Finally, remove the value that was pre-populated for parDdosProtectionPlanId, otherwise you'll continue to run into the same issue deploying other Virtual Networks.
Apologies for the confusion, and we will work on providing clarification within the documentation for this scenario and any others that could be impacted by the built-in. We will also consider adding an additional input request to the ALZ-PowerShell-Module to determine if the DDoS protection should be enabled or not.
Hi @oZakari,
This did the trick. Thank you.
Just would like to add that this policy assignment "Virtual networks should be protected by Azure DDoS Protection Standard" is added on two levels: "Landing Zones" and "Connectivity", so if someone wants to remove them manually, needs to remove on both levels.
Thanks!
Or you can add them to this array parameter parExcludedPolicyAssignments
I just ran into this as well while using the ALZ accelerator pipelines using Azure DevOps.
There you have to add it to config\custom-parameters\alzDefaultPolicyAssignments.parameters.all.json
"parExcludedPolicyAssignments": {
"value": [
"Enable-DDoS-VNET"
]
},
A possible solution in the future might be when the parameter files are refactored from json
- to bicepparam
-files, so that a description or comment can be added with the parameter, for example:
@sys.description('Switch to enable/disable DDoS Network Protection deployment. When you set this to false, make sure you add the folowing policy to the parExcludedPolicyAssignments in the alzDefaultPolicyAssignments.parameters.all parameter file: "Enable-DDoS-VNET" ')
param parDdosEnabled bool = true
@MarcoJanse fix works only if you didn't already deploy the Azure Policy, but if already deployed, you will have to go into the Azure Policy and change enforcement mode from enabled
to disabled
as outlined in the docs.
Issue:
If you update a policy or version, it will revert back to default
, because of infra-as-code\bicep\modules\policy\assignments\lib\policy_assignments\policy_assignment_es_enable_ddos_vnet.tmpl.json
. Same goes for doing what @oZakari states by removing it from the connection management group. It will simply appear back. Since the Hub is already created, it won't get triggered, but also won't be compliant.
Workaround
Because of what is explained above the workaround would be to first go into the Azure Policy and disable it, then go into config\custom-parameters\alzDefaultPolicyAssignments.parameters.all.json
and add the parExcludedPolicyAssignments.value = "Enable-DDoS-VNET"
. This will make it so that that Enable-DDoS-VNET
Policy simply will be ignored.
The suggested future fix that @MarcoJanse shared, would be ideal as it keeps with the opt-out function of deploying DDoS.
I've documented the workaround described here by MarcoJanse and FallenHoot https://github.com/Azure/ALZ-Bicep/pull/711/files
As an alternative workaround, can we change the two policy definitions enforcementMode
from Default
to DoNotEnforce
?
definitions are in the files
upstream-releases\v0.17.2\infra-as-code\bicep\modules\policy\assignments\lib\policy_assignments\policy_assignment_es_enable_ddos_vnet.tmpl.json
and
upstream-releases\v0.17.2\infra-as-code\bicep\modules\policy\assignments\lib\china\policy_assignments\policy_assignment_es_enable_ddos_vnet.tmpl.json
The PR mentioned above will also add a parameter parDdosEnabled
to the the assignment to DoNotEnforce as well so you don't have to mess with the policy assignment json files directly if you don't want.
Hello,
I am having trouble with running the ALZ-Bicep-4a Workflow component.
I have edited the config/custom-parameters/hubNetworking.parameters.all.json file to exclude the DDoS protection as following:
However, when I run the deployment, it fails with the following error:
Resource /subscriptions/connectivitysubID/resourceGroups/rgname/providers/Microsoft.Network/ddosProtectionPlans/alz-ddos-plan referenced by resource /subscriptions/connectivitysubID/resourceGroups/rgname/providers/Microsoft.Network/virtualNetworks/vnet-name was not found. Please make sure that the referenced resource exists.
Raw error from deployment:
To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
The workflow should not create DDOS Protection and successfully complete.
Screenshots 📷
Correlation ID
c6905a2e-8408-4c9a-96fe-172dc2390a3a
Additional context
Using Accelerator v0.16.0.
config/custom-parameters/hubNetworking.parameters.all.json:
Pipeline:
Pipeline script: