Closed khabbabs closed 2 years ago
Hi @khabbabs... thank you for this interesting request. We've actually discussed similar scenarios with other customers and already have an answer for this, although we should probably make this clearer in the broader Enterprise-scale documentation.
In short, the design was intentionally meant to have a 1:1 relationship between each of the platform Management Groups and their corresponding Subscriptions due to the way some policies were written. I'm not sure if this is so critical now (this was mainly relating to connectivity policies with IPAM parameters which have since been removed) but we still recommend that you do not use Management Groups to differentiate between environments.
That said, if you really want to do something like this, whilst retaining functionality of the management resources you will need to ensure your management
Subscription lands in your management
Management Group as this is how the module ensures all the resources are in the correct place.
Although I wouldn't recommend this, to achieve your desired hierarchy you can use the following as a baseline for your module config:
module "enterprise_scale" {
source = "Azure/caf-enterprise-scale/azurerm"
version = "1.0.0"
providers = {
azurerm = azurerm.management
azurerm.connectivity = azurerm.management
azurerm.management = azurerm.management
}
root_parent_id = data.azurerm_client_config.management.tenant_id
root_id = local.root_id
root_name = local.root_name
custom_landing_zones = {
# Create a new base Management Group under platform for `Management`
# Must have a different ID such as below
"${local.root_id}-management-base" = {
display_name = "Management"
parent_management_group_id = "${local.root_id}-platform"
subscription_ids = []
archetype_config = {
# Map this to the default_empty archetype definition
archetype_id = "default_empty"
parameters = {}
access_control = {}
}
}
# Redefine the built-in `management` Management Group
# Set the display_name to your desired value of `prod`
# Set the parent_management_group_id to the ID of the MG above
"${local.root_id}-management" = {
display_name = local.env # I assume this maps correctly to your desired value of `prod`
parent_management_group_id = "${local.root_id}-management-base"
subscription_ids = [
# Need to manually map management.subscription_id as the built-in logic is overriden by this config
data.azurerm_client_config.management.subscription_id,
]
archetype_config = {
# Make sure you map this to the original es_management archetype definition
archetype_id = "es_management"
parameters = {}
access_control = {}
}
}
}
deploy_management_resources = true
configure_management_resources = local.configure_management_resources
subscription_id_management = data.azurerm_client_config.management.subscription_id
}
This should give you the desired structure as below, whilst minimising the impact on integration functionality.
Please let me know how you get on!
Also just to chip in here. I would recommend a review of https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/testing-approach that may be more suited to your requirements and change the hierarchy 👍
Do you still need to set deploy_core_landing_zones as "true", or does this need to be changed to false to deploy it this way Kevin?
Also just to chip in here. I would recommend a review of https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/testing-approach that may be more suited to your requirements and change the hierarchy 👍
Thanks for this. How do you ensure logs from the canary still get distributed to the primary log analytics workspace, instead of provisioning new for, "Send all Azure activity logs for all Azure subscriptions, including any canary environment subscriptions, to the production environment Azure Log Analytics workspace as per the enterprise-scale design recommendations."
Do you still need to set deploy_core_landing_zones as "true", or does this need to be changed to false to deploy it this way Kevin?
In the example above, the assumption is to have deploy_core_landing_zones = true
which is the default value anyway.
If you don't want the remaining hierarchy you can set this to false
but will need to build the entire hierarchy within custom_landing_zones
Also just to chip in here. I would recommend a review of https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/testing-approach that may be more suited to your requirements and change the hierarchy 👍
Thanks for this. How do you ensure logs from the canary still get distributed to the primary log analytics workspace, instead of provisioning new for, "Send all Azure activity logs for all Azure subscriptions, including any canary environment subscriptions, to the production environment Azure Log Analytics workspace as per the enterprise-scale design recommendations."
Great question... we'll come back to you on this.
cc: @jtracey93
Also just to chip in here. I would recommend a review of https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/testing-approach that may be more suited to your requirements and change the hierarchy 👍
Thanks for this. How do you ensure logs from the canary still get distributed to the primary log analytics workspace, instead of provisioning new for, "Send all Azure activity logs for all Azure subscriptions, including any canary environment subscriptions, to the production environment Azure Log Analytics workspace as per the enterprise-scale design recommendations."
Great question... we'll come back to you on this.
cc: @jtracey93
In terms of this module (@krowlandson correct me if I'm wrong here 👍) you would need to just ensure the parameter value passed into the policy assignment in your canary deployment for Deploy-AzActivity-Log
is set to the "production" log analytics workspace. You can do this with an archetype_config_overrides
as below:
archetype_config_overrides = {
root = {
archetype_id = "es_root"
parameters = {
Deploy-AzActivity-Log = {
logAnalytics = "/subscriptions/11111111-1111-1111-1111-000000000000/resourcegroups/test-mgmt/providers/Microsoft.OperationalInsights/workspaces/test-la"
}
}
access_control = {}
}
}
Replace the resource ID with your own
You need to ensure the following:
root_id
is unique between prod and canary environmentsarchetype_config_overrides
as above to change the input value for the Deploy-AzActivity-Log
policy in your canary environmentHey @krowlandson in your example how did you make sure it only deploys one management group? I followed your example and i'm getting two different management groups, one from the custom landing zones "myorg-management-base" and another which is part of the core landing zones "myorg-management".
Here's the custom_landing_zones code
custom_landing_zones = local.deploy_custom_landing_zones ? {
"${local.root_id}-management-base" = {
display_name = "Management"
parent_management_group_id = "${local.root_id}-platform"
subscription_ids = []
archetype_config = {
archetype_id = "default_empty"
parameters = {}
access_control = {}
}
}
"${local.root_id}-management-prod" = {
display_name = "prod"
parent_management_group_id = "${local.root_id}-management-base"
subscription_ids = [
data.azurerm_client_config.management.subscription_id,
]
archetype_config = {
archetype_id = "es_management"
parameters = {}
access_control = {}
}
}
"${local.root_id}-management-nonprod" = {
display_name = "nonprod"
parent_management_group_id = "${local.root_id}-management-base"
subscription_ids = []
archetype_config = {
archetype_id = "es_management"
parameters = {}
access_control = {}
}
}
deploy_management_resources = true
configure_management_resources = local.configure_management_resources
subscription_id_management = data.azurerm_client_config.management.subscription_id
Hey @krowlandson in your example how did you make sure it only deploys one management group? I followed your example and i'm getting two different management groups, one from the custom landing zones "myorg-management-base" and another which is part of the core landing zones "myorg-management".
Here's the custom_landing_zones code
custom_landing_zones = local.deploy_custom_landing_zones ? { "${local.root_id}-management-base" = { display_name = "Management" parent_management_group_id = "${local.root_id}-platform" subscription_ids = [] archetype_config = { archetype_id = "default_empty" parameters = {} access_control = {} } } "${local.root_id}-management-prod" = { display_name = "prod" parent_management_group_id = "${local.root_id}-management-base" subscription_ids = [ data.azurerm_client_config.management.subscription_id, ] archetype_config = { archetype_id = "es_management" parameters = {} access_control = {} } } "${local.root_id}-management-nonprod" = { display_name = "nonprod" parent_management_group_id = "${local.root_id}-management-base" subscription_ids = [] archetype_config = { archetype_id = "es_management" parameters = {} access_control = {} } } deploy_management_resources = true configure_management_resources = local.configure_management_resources subscription_id_management = data.azurerm_client_config.management.subscription_id
Hey @khabbabs I think this comment covers this off https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/issues/200#issuecomment-962022810
👍
Hey @krowlandson Kevin Rowlandson FTE in your example how did you make sure it only deploys one management group? I followed your example and i'm getting two different management groups, one from the custom landing zones "myorg-management-base" and another which is part of the core landing zones "myorg-management".
The critical part here is re-defining the built-in Management Group definition for ${local.root_id}-management
. In your example, you and a suffix to all of the Management Group IDs, so the default one will still be deployed and then used as the target for all module logic.
This means you would effectively have the following management
Management Groups:
ID | Display Name | Parent |
---|---|---|
${local.root_id}-management-base |
Management |
${local.root_id}-platform |
${local.root_id}-management |
prod |
${local.root_id}-management-base |
${local.root_id}-management-nonprod |
nonprod |
${local.root_id}-management-base |
As an aside, it would be interesting to understand your use-case as environment aligned Management Groups is actually an ES anti-pattern. We also recommend only implementing a single Log Analytics workspace for all logs. It will be very challenging to correctly configure Policy Assignments to ensure things like Activity Logs and Diagnostics go to the correct Log Analytics workspace, but using multiple Log Analytics workspaces will also greatly increase the complexities associated with running queries across all Subscriptions and integration with external tooling (e.g. SIEM, etc.)
@krowlandson and @jtracey93 thank you so much for all your help and suggestions. At this moment we're still experimenting with what we want to do.
I had another question regarding custom landing zones. How does the arche type exclusion work with custom landing zones?
for example.
I have a file archetype_exclusion_es_identity.tmpl.json
{
"exclude_es_identity": {
"policy_assignments": [
"Deny-Subnet-Without-NSG"
],
"policy_definitions": [],
"policy_set_definitions": [],
"role_definitions": [],
"archetype_config": {
"parameters": {},
"access_control": {}
}
}
}
Does this exclusion get inherited to the custom landing zones which have "es_identity" listed in the archetype_id...
"${local.root_id}-identity-prod" = {
display_name = "prod"
parent_management_group_id = "${local.root_id}-identity"
subscription_ids = ["${var.subscription_id_identity}"]
archetype_config = {
archetype_id = "es_identity"
parameters = {}
access_control = {}
}
}
Hey @khabbabs,
The exclusion feature works against the archetype definitions themselves, so are agnostic to custom landing zones etc.
So you may find the case where if you need to different exclusion from the same archetype definition on different management groups, you may actually need to create separate archetype definitions to achieve this 👍 (which aligns with archetypes - as they will be different 😃)
Hope that helps
Jack
Thanks for all your help! if we have any more questions, i'll review the docs before opening up another issue if need be.
Community Note
Description
Not sure if this is an a feature request but I do need some help in figuring out if something like this is possible. I'm trying to keep the recommended hierarchy but with an additional layer. Basically keeping the core landing zones but with another layer added.
Something like
Ignore the nonprod management group. I want to take the core management connectivity and identity groups under another layer. Really hope the pic helps describe what i'm wanting.
Is your feature request related to a problem?
I can create a similar hierarchy using the custom landing zones but I can't deploy all resources for it that come with the core management groups.
Any suggestions would be appreciated.
Describe the solution you'd like
I would like to create a custom landing zone under the parent management group like ...
at the same time I would like to deploy the resources generated from settings.mangement.tf
the main file has this for the management resource
This keeps the subscription under the parent management group and no the prod management group that I created in the custom landing zones section.
Please let me know if you need any more clarification! Really love the solution you already have, I would just like to make some small adjustments.
Additional context
TBD