Unable to maintain DNS Zone Virtual Network Links

JackBruceShell commented 3 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

Latest

AzureRM Provider Version

3.95.0

Affected Resource(s)/Data Source(s)

azurerm_private_dns_zone_virtual_network_link

Terraform Configuration Files

resource "azurerm_app_service_environment_v3" "main" {
  name     = var.ase_name
  resource_group_name = var.resource_group_name
  subnet_id = var.ase_subnet
  internal_load_balancing_mode = "Web, Publishing"
  zone_redundant = var.zone_redundant
  cluster_setting  {
    name = "DisableTls1.0"
    value = "1"
  }
  tags = {
    businessunit         = var.businessunit
  }
}

resource "azurerm_private_dns_zone" "main" {
  count               = var.environment == "External" ? 1 : 0
  name                = format("%s.%s",var.ase_name,"appserviceenvironment.net")
  resource_group_name = azurerm_app_service_environment_v3.main.resource_group_name
}

resource "azurerm_private_dns_a_record" "record_1" {
  count               = var.environment == "External" ? 1 : 0
  name                = "*"
  zone_name           = azurerm_private_dns_zone.main[count.index].name
  resource_group_name = azurerm_private_dns_zone.main[count.index].resource_group_name
  ttl                 = 3600
  records             = azurerm_app_service_environment_v3.main.internal_inbound_ip_addresses 
}

resource "azurerm_private_dns_a_record" "record_2" {
  count               = var.environment == "External" ? 1 : 0
  name                = "@"
  zone_name           = azurerm_private_dns_zone.main[count.index].name
  resource_group_name = azurerm_private_dns_zone.main[count.index].resource_group_name
  ttl                 = 3600
  records             = azurerm_app_service_environment_v3.main.internal_inbound_ip_addresses 
}

resource "azurerm_private_dns_a_record" "record_3" {
  count               = var.environment == "External" ? 1 : 0
  name                = "*.scm"
  zone_name           = azurerm_private_dns_zone.main[count.index].name
  resource_group_name = azurerm_private_dns_zone.main[count.index].resource_group_name
  ttl                 = 3600
  records             = azurerm_app_service_environment_v3.main.internal_inbound_ip_addresses 
}

data "azurerm_virtual_network" "main" {
  count               = var.environment == "External" ? 1 : 0
  name                = var.vnet_name
  resource_group_name = var.vnet_rg_name
}

resource "azurerm_private_dns_zone_virtual_network_link" "main" {
  count               = var.environment == "External" ? 1 : 0
  name                  = "vnetlink"
  resource_group_name   = azurerm_app_service_environment_v3.main.resource_group_name
  private_dns_zone_name = azurerm_private_dns_zone.main[count.index].name
  virtual_network_id    = data.azurerm_virtual_network.main[count.index].id
}

resource "azurerm_monitor_diagnostic_setting" "main" {
  count = var.enable_diagnostic ? 1 : 0
  name               = "${azurerm_app_service_environment_v3.main.name}-diagnostic"
  target_resource_id = azurerm_app_service_environment_v3.main.id
  log_analytics_workspace_id = var.log_analytics_workspace_id

  enabled_log {
    category = "AppServiceEnvironmentPlatformLogs"
    retention_policy {
      enabled = true
      days = 0
    }
  }
  metric {
    category = "AllMetrics"
    retention_policy {
      enabled = true
      days = 0
    }
  }
}

Debug Output/Panic Output

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
 <= read (data resources)

Terraform will perform the following actions:

  # module.demo-nsg1.azurerm_network_security_group.main will be updated in-place
  ~ resource "azurerm_network_security_group" "main" {
        id                  = "/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-APIM-NSD-001"
        name                = "NSG-AIS-ENT-APIM-NSD-001"
      ~ tags                = {
          - "COB"          = "NSD" -> null
          - "LOB"          = "NSD" -> null
          - "Project Name" = "NSD" -> null
          - "Sold to Code" = "1234567" -> null
            "businessunit" = ""
        }
        # (3 unchanged attributes hidden)
    }

  # module.demo-nsg2.azurerm_network_security_group.main will be updated in-place
  ~ resource "azurerm_network_security_group" "main" {
        id                  = "/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-ASE-NSD-001"
        name                = "NSG-AIS-ENT-ASE-NSD-001"
      ~ tags                = {
          - "COB"          = "NSD" -> null
          - "LOB"          = "NSD" -> null
          - "Project Name" = "NSD" -> null
          - "Sold to Code" = "1234567" -> null
            "businessunit" = ""
        }
        # (3 unchanged attributes hidden)
    }

  # module.demo-vnet.azurerm_virtual_network.main will be updated in-place
  ~ resource "azurerm_virtual_network" "main" {
        id                      = "/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/virtualNetworks/VNET-AIS-NSD-10.0.0.0-16"
        name                    = "VNET-AIS-NSD-10.0.0.0-16"
      ~ tags                    = {
          - "COB"          = "NSD" -> null
          - "LOB"          = "NSD" -> null
          - "Project Name" = "NSD" -> null
          - "Sold to Code" = "1234567" -> null
            "businessunit" = ""
        }
        # (7 unchanged attributes hidden)
    }

      + name                         = "ASP-AIS-ENT-NSD-001"
      + os_type                      = "Windows"
      + per_site_scaling_enabled     = false
      + reserved                     = (known after apply)
      + resource_group_name          = "RG-AIS-ENT-DEV-NSD-002"
      + sku_name                     = "I1v2"
      + worker_count                 = (known after apply)
      + zone_balancing_enabled       = true
    }

Plan: 6 to add, 4 to change, 0 to destroy.
module.demo-nsg2.azurerm_network_security_group.main: Modifying... [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-ASE-NSD-001]
module.demo-nsg1.azurerm_network_security_group.main: Modifying... [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-APIM-NSD-001]
module.demo-nsg2.azurerm_network_security_group.main: Modifications complete after 1s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-ASE-NSD-001]
module.demo-nsg1.azurerm_network_security_group.main: Modifications complete after 1s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/networkSecurityGroups/NSG-AIS-ENT-APIM-NSD-001]
module.demo-vnet.azurerm_virtual_network.main: Modifying... [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/virtualNetworks/VNET-AIS-NSD-10.0.0.0-16]
module.demo-vnet.azurerm_virtual_network.main: Modifications complete after 3s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/virtualNetworks/VNET-AIS-NSD-10.0.0.0-16]
module.nsd-ase.data.azurerm_virtual_network.main[0]: Reading...
module.nsd-ase.azurerm_app_service_environment_v3.main: Modifying... [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Web/hostingEnvironments/ASE-AIS-ENT-DEV-NSD-001]
module.nsd-ase.data.azurerm_virtual_network.main[0]: Read complete after 1s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/virtualNetworks/VNET-AIS-NSD-10.0.0.0-16]
module.nsd-ase.azurerm_app_service_environment_v3.main: Still modifying... [id=/subscriptions/afeae369-3a58-4f5c-a4bc-...ngEnvironments/ASE-AIS-ENT-DEV-NSD-001, 10s elapsed]
module.nsd-ase.azurerm_app_service_environment_v3.main: Still modifying... [id=/subscriptions/afeae369-3a58-4f5c-a4bc-...ngEnvironments/ASE-AIS-ENT-DEV-NSD-001, 20s elapsed]
module.nsd-ase.azurerm_app_service_environment_v3.main: Still modifying... [id=/subscriptions/afeae369-3a58-4f5c-a4bc-...ngEnvironments/ASE-AIS-ENT-DEV-NSD-001, 30s elapsed]
module.nsd-ase.azurerm_app_service_environment_v3.main: Modifications complete after 33s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Web/hostingEnvironments/ASE-AIS-ENT-DEV-NSD-001]
module.nsd-ase.azurerm_private_dns_zone.main[0]: Creating...
module.nsd-ase.azurerm_private_dns_zone.main[0]: Still creating... [10s elapsed]
module.nsd-ase.azurerm_private_dns_zone.main[0]: Still creating... [20s elapsed]
module.nsd-ase.azurerm_private_dns_zone.main[0]: Still creating... [30s elapsed]
module.nsd-ase.azurerm_private_dns_zone.main[0]: Creation complete after 33s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/privateDnsZones/ASE-AIS-ENT-DEV-NSD-001.appserviceenvironment.net]
module.nsd-ase.azurerm_private_dns_a_record.record_1[0]: Creating...
module.nsd-ase.azurerm_private_dns_a_record.record_2[0]: Creating...
module.nsd-ase.azurerm_private_dns_a_record.record_3[0]: Creating...
module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0]: Creating...
module.nsd-ase.azurerm_private_dns_a_record.record_2[0]: Creation complete after 1s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/privateDnsZones/ASE-AIS-ENT-DEV-NSD-001.appserviceenvironment.net/A/@]
module.nsd-ase.azurerm_private_dns_a_record.record_1[0]: Creation complete after 2s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/privateDnsZones/ASE-AIS-ENT-DEV-NSD-001.appserviceenvironment.net/A/*]
module.nsd-ase.azurerm_private_dns_a_record.record_3[0]: Creation complete after 2s [id=/subscriptions/***/resourceGroups/RG-AIS-ENT-DEV-NSD-002/providers/Microsoft.Network/privateDnsZones/ASE-AIS-ENT-DEV-NSD-001.appserviceenvironment.net/A/*.scm]
module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0]: Still creating... [10s elapsed]
module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0]: Still creating... [20s elapsed]
module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0]: Still creating... [30s elapsed]

Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0],
│ provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced
│ an unexpected new value: Root object was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

Expected Behaviour

No issues with Virtual Link resource when trying to provision other resources.

Actual Behaviour

Terraform apply failed with error

Error: Provider produced inconsistent result after apply │ │ When applying changes to │ module.nsd-ase.azurerm_private_dns_zone_virtual_network_link.main[0], │ provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced │ an unexpected new value: Root object was present, but now absent. │ │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker.

Steps to Reproduce

terraform apply

Unique use case and can present the issue on a call if needed.

Important Factoids

No response

References

No response

neil-yechenwei commented 3 months ago

Thanks for raising this issue. Could you try to add "depends_on" to explicitly build the dependency among the resources in the tf config to see if the issue still exists? Thanks.

Arsur commented 3 months ago

depends_on should not be needed, because there is already a depency between the resources

For me a roleback to version 3.92 solved this problem, i did not test already which of the higher versions work. Sometime i also had this error message with a user managed identity. Maybe its something with the new sdk versions

UPDATE: Version 3.92 did not solve this problem. I have it as well!

emad0082 commented 3 months ago

I am also having issues regards to virtual network links.

Issue: Inconsistent Results with azurerm_private_dns_zone_virtual_network_link

Description: Encountering issues creating/destroying Virtual Link resources in Azure using azurerm provider version ~>3.75.0. Error message received:

Error: Provider produced inconsistent result after apply

When applying changes to azurerm_private_dns_zone_virtual_network_link.kv-vlink, provider "provider["registry.terraform.io/hashicorp/azurerm"].networkhub-subscription" produced an unexpected new value: Root resource was present, but now absent.

Code:


data "azurerm_private_dns_zone" "kv_private_dns" {
  provider            = azurerm.networkhub-subscription
  name                = "privatelink.vaultcore.azure.net"
  resource_group_name = "RGH-P-PrivateDNSZones-rg"
}

Additional Info:

Provider: registry.terraform.io/hashicorp/azurerm
Provider Version: ~>3.75.0
This seems to be a provider bug. Any insights or assistance on resolving this issue would be appreciated.

Arsur commented 3 months ago

We did some further investigation. We get this error only on at Region West Europe. Running the same code in Germany West Central worked perfectly. Due to the fact that region west europe is under heavy duty and almost full this may be hard to fix

dsczltch commented 1 month ago

We also encounter this issue. Terraform version : 1.8.4 AzureRM version : 3.90.0 Affected Resource(s)/Data Source(s) : azurerm_private_dns_zone_virtual_network_link Region : france central

Our code : (##) link dns zone private with hub vnet resource "azurerm_private_dns_zone_virtual_network_link" "hub-dnszone-network_links" { provider = azurerm.hub

name = "hub-dnszone-vnetlink" resource_group_name = "hub-network-rg" private_dns_zone_name = azurerm_private_dns_zone.hub-private-dns-zones.name virtual_network_id = data.azurerm_virtual_network.hub-network-vnet.id tags = local.tags_api }

data "azurerm_virtual_network" "hub-network-vnet" { provider = azurerm.hub

name = "hub-network-vnet" resource_group_name = "hub-network-rg" }

The output :

│ Error: Provider produced inconsistent result after apply │ │ When applying changes to │ module.container_app_environment.azurerm_private_dns_zone_virtual_network_link.hub-dnszone-network_links, │ provider │ "module.container_app_environment.provider[\"registry.terraform.io/hashicorp/azurerm\"].hub" │ produced an unexpected new value: Root object was present, but now absent. │ │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker. ╵ ╷ │ Error: creating/updating Virtual Network Link (Subscription: "57a2cf7c-e2c6-429f-95b7-77ca665522bb" │ Resource Group Name: "hub-network-rg" │ Private Dns Zone Name: "politesea-xxxxxxxx.francecentral.azurecontainerapps.io" │ Virtual Network Link Name: "hub-dnszone-vnetlink"): performing CreateOrUpdate: unexpected status 409 with response: {"code":"Conflict","message":"Another operation is pending for requested object. Operation group '\/operations\/groups\/id\/|virtualNetworkLinks|57a2cf7c-e2c6-429f-95b7-77ca665522bb|hub-network-rg|politesea-xxxxxxxx.francecentral.azurecontainerapps.io|hub-dnszone-vnetlink' was created simultaneously by another operation."} │ │ with module.container_app_environment.azurerm_private_dns_zone_virtual_network_link.env-dnszone-default-network-links, │ on ../../../infrastructure-tf-modules/azurerm/container_app_environment/main.tf line 77, in resource "azurerm_private_dns_zone_virtual_network_link" "env-dnszone-default-network-links": │ 77: resource "azurerm_private_dns_zone_virtual_network_link" "env-dnszone-default-network-links" { │ ╵

The DNS virtual network link is created in Azure but not added to the state file.

Could anyone please have a look to this issue and create a fix? Thank you in advance

hashicorp / terraform-provider-azurerm