[AVM Module Issue]: Missing private_dns_zone_id resource definition

0xtvignoli commented 3 weeks ago

Check for previous/existing GitHub issues

[x] I have checked for previous/existing GitHub issues

Issue Type?

Feature Request

(Optional) Module Version

0.2.0

(Optional) Correlation Id

No response

Description

When applying the terraform-azurerm-avm-ptn-aks-production module at version 0.2.0, Terraform attempts to provision a new AKS cluster. This includes various resource definitions within azurerm_kubernetes_cluster.this triggering a provisioning process with the following parameters:

resource "azurerm_kubernetes_cluster" "this" {
    api_server_authorized_ip_ranges     = (known after apply)
    automatic_channel_upgrade           = "patch"
    azure_policy_enabled                = true
    ...
    private_dns_zone_id                 = (known after apply)
    ...
}

Issue Encountered

The module fails due to an enterprise-level Landing Zone (LZ) policy restriction on private_dns_zone_id creation within azurerm_kubernetes_cluster.this. Specifically, the policy prohibits the creation of private DNS zones, returning the following error upon apply:

Status: "CreateOrUpdatePrivateZoneFailed"
Code: "RequestDisallowedByPolicy"
Message: "Create or update private DNS zone failed. 
Resource 'XXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.privatelink.westeurope.azmk8s.io' was disallowed by policy.
Policy identifiers: '[{\"policyAssignment\": ... \"policyAssignmentDisplayName\":\"Deny the creation of private DNS\", ... \"policyDefinitionDisplayName\":\"Deny the creation of private DNS\"}]'

The policy originates from enterprise-level management configurations, specifically aimed at restricting private DNS resource creation across the subscription's management group.

To summarize:

Enterprise-level policy restricts private DNS zone creation across the subscription's management group
The module doesn't provide a way to handle pre-existing private DNS zones or make the private DNS zone creation optional
This conflicts with enterprise governance policies that centralize DNS management

Proposed Solution

Add configuration options to:

Accept an existing private_dns_zone_id as an input variable
Make private DNS zone creation optional through a feature flag
Document enterprise DNS governance considerations

Environment Details

Module Version: 0.2.0
Terraform Version: 1.9.1
Provider: AzureRM:3.116.0
Deployment Context: Enterprise Landing Zone

zioproto commented 3 weeks ago

Setting the expectation: This module is not compatible with enterprise level Landing Zone. This was never the goal.

The proposed solution does not work because it seems to me the Terraform provider accepts only the values System or None

https://github.com/hashicorp/terraform-provider-azurerm/blob/ecede1a79160d07d4a294794496663ae9dcb9220/internal/services/containers/kubernetes_cluster_resource.go#L1284-L1296

From the AKS documentation it seems that in Azure CLI --private-dns-zone parameter can actually accept a resourceID value in addition to System and None

https://learn.microsoft.com/en-us/azure/aks/private-clusters?tabs=default-basic-networking%2Cazure-portal#create-a-private-aks-cluster-with-a-custom-private-dns-zone-or-private-dns-subzone

But looking at azure-rest-api-specs again I see in the description only System or None

https://github.com/Azure/azure-rest-api-specs/blob/065bac6153f6a83de668f55803270bc6f31f5bc3/specification/containerservice/resource-manager/Microsoft.ContainerService/aks/stable/2024-08-01/managedClusters.json#L5023-L5027

@ms-henglu can you please have a look to check if we are eventually missing a feature in the provider code ? thanks

zioproto commented 3 weeks ago

I actually read the code again: I missed privatezones.ValidatePrivateDnsZoneID:

https://github.com/hashicorp/terraform-provider-azurerm/blob/ecede1a79160d07d4a294794496663ae9dcb9220/internal/services/containers/kubernetes_cluster_resource.go#L1290C19-L1290C43

@0xtvignoli making private_dns_zone_id configurable to a PrivateDnsZoneID would fix your problem ? I want to avoid the possibility of setting it to None

0xtvignoli commented 3 weeks ago

Thank you for the quick response. I believe the proposed solution does not align with the current as-is infrastructure, as private DNS zones can only be managed within the Landing Zone (LZ) framework, rather than directly at the subscription level.

zioproto commented 1 week ago

I need more eyes on this issue because I am not sure what is the right thing to do:

@matt-FFFFFF @lonegunmanb @nellyk @ibersanoms @mosabami I don't know how to build a test scenario for this.

If I create a variable for private_dns_zone_id letting the user specify a DNS zone ID that lives in a different subscription, how can I test this scenario with the AVM CI ?

Do we have other AVM modules that need to support DNS zones in a different subscription ? Do we have any other AVM module supporting Landing Zones scenarios ?

thanks

matt-FFFFFF commented 1 week ago

Hi,

You can use an example. Deploy the dns zone in the example root. Then pass in the resource id into the module

zioproto commented 1 week ago

@matt-FFFFFF But how can I create the DNS zone in a different subscription than the one where I deploy the module ?

matt-FFFFFF commented 1 week ago

You can't but that won't matter for the purposes of the example. It will demonstrate that the module can take in a dependency.

zioproto commented 1 week ago

@0xtvignoli PR #139 should solve the problem of setting private_dns_zone_id and should get you unblocked with the Landing Zone. Please let me know if the PR looks good to you. thanks

0xtvignoli commented 1 day ago

Thanks @zioproto for you effort! 👍

Azure / terraform-azurerm-avm-ptn-aks-production