Existing associations prevent data collection rule from being deleted

[msapitree]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.7

AzureRM Provider Version

v3.88.0

Affected Resource(s)/Data Source(s)

azurerm_monitor_data_collection_rule

Terraform Configuration Files

resource "azurerm_monitor_data_collection_rule" "insights" {
  name                        = "dcr-${var.aks_name}"
  resource_group_name         = local.resource_group_name
  location                    = local.location
  data_collection_endpoint_id = var.data_collection_endpoint_id
  tags                        = local.merged_tags

  data_sources {
    extension {
      name           = "ContainerInsightsExtension"
      streams        = ["Microsoft-ContainerInsights-Group-Default"]
      extension_name = "ContainerInsights"
      extension_json = jsonencode({
        dataCollectionSettings = {
          interval               = "5m"
          namespaceFilteringMode = "Exclude"
          namespaces             = ["kube-system", "gatekeeper-system", "azure-arc"]
          enableContainerLogV2   = true
        }
      })
    }
  }

  data_flow {
    streams      = ["Microsoft-ContainerInsights-Group-Default"]
    destinations = [data.azurerm_log_analytics_workspace.this.name]
  }

  destinations {
    log_analytics {
      name                  = data.azurerm_log_analytics_workspace.this.name
      workspace_resource_id = data.azurerm_log_analytics_workspace.this.id
    }
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.aks_data_collector.id]
  }
}

resource "azurerm_monitor_data_collection_rule_association" "aks" {
  name                    = "dcra-${var.aks_name}"
  target_resource_id      = azurerm_kubernetes_cluster.this.id
  data_collection_rule_id = azurerm_monitor_data_collection_rule.insights.id
}

// -----------------------------------------------------------------------------
// CALCULATED PLAN
// Note: the DCR is planned to be replaced (actually, there is no change in the location, but it's the other issue)
// -----------------------------------------------------------------------------
  # module.aks.azurerm_monitor_data_collection_rule.insights must be replaced
-/+ resource "azurerm_monitor_data_collection_rule" "insights" {
      ~ id                          = "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.Insights/dataCollectionRules/dcr-<aks_name>" -> (known after apply)
      ~ immutable_id                = "dcr-fa7f6c69b41145bfa7885e673396a5fc" -> (known after apply)
      ~ location                    = "westeurope" # forces replacement -> (known after apply) # forces replacement
        name                        = "dcr-aks-<aks_name>"
      - tags                        = {} -> null
        # (2 unchanged attributes hidden)
      ~ data_sources {
          ~ extension {
              - input_data_sources = [] -> null
                name               = "ContainerInsightsExtension"
                # (3 unchanged attributes hidden)
            }
        }
      ~ destinations {
          ~ log_analytics {
                name                  = "law-<aks_name>"
              ~ workspace_resource_id = "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.OperationalInsights/workspaces/law-<aks_name>" -> (known after apply)
            }
        }
      ~ identity {
          ~ identity_ids = [
              - "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-dc-<aks_name>",
            ] -> (known after apply)
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
            # (1 unchanged attribute hidden)
        }
        # (1 unchanged block hidden)
    }
  # module.aks.azurerm_monitor_data_collection_rule_association.aks will be updated in-place
  ~ resource "azurerm_monitor_data_collection_rule_association" "aks" {
      ~ data_collection_rule_id = "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.Insights/dataCollectionRules/dcr-<aks_name>" -> (known after apply)
        id                      = "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.ContainerService/managedClusters/<aks_name>/providers/Microsoft.Insights/dataCollectionRuleAssociations/dcra-<aks_name>"
        name                    = "dcra-<aks_name>"
        # (1 unchanged attribute hidden)
    }

Debug Output/Panic Output

all fine

Expected Behaviour

azurerm_monitor_data_collection_rule should be successfully replaced, i.e. no error on terraform apply

If it means the resource has to be destroyed first, implying any association has to be destroyed too , then any such association should be deleted first and then, once the DCR is recreated, the association should be recreated too.

Actual Behaviour

│ Error: deleting Data Collection Rule (Subscription: "<subscription_id>"
│ Resource Group Name: "<rg_name>"
│ Data Collection Rule Name: "dcr-<cluster_name>"): datacollectionrules.DataCollectionRulesClient#Delete: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="ExistingAssociationsPreventDelete" Message="Existing associations prevent data collection rule from being deleted: \r\n /subscriptions/<subscription_id>/resourcegroups/<rg_name>/providers/microsoft.containerservice/managedclusters/<cluster_name>/providers/microsoft.insights/datacollectionruleassociations/dcra-<cluster_name>" Details=[{"code":"ExistingAssociationsPreventDelete","message":"Existing associations prevent data collection rule from being deleted: \r\n /subscriptions/<subscription_id>/resourcegroups/<rg_name>/providers/microsoft.containerservice/managedclusters/<cluster_name>/providers/microsoft.insights/datacollectionruleassociations/dcra-<cluster_name>"}]

Note: Resource Group, Subscription etc. are redacted

Steps to Reproduce

No response

Important Factoids

No response

References

No response

[teowa]

Hi @msapitree , thanks for submitting this! I can successfully reproduce the issue by changing the location property of an existing data collection rule.

The expected Terraform behavior of this case is first detroy the azurerm_data_collection_rule_association, and then destroy and re-create the azurerm_data_collection_rule, finally recreate the association. But this is not happen because Terraform only want to update the association in place because the azurerm_data_collection_rule_association.data_collection_rule_id is not forceNew field. But from test, the rule accociated can be changed from one to another, so we didn't regard this as a forceNew field.

A workaround for this is to add replace_triggered_by for the azurerm_data_collection_rule_association:

resource "azurerm_monitor_data_collection_rule_association" "example" {
  name                    = "example-dcra"
  target_resource_id      = azurerm_kubernetes_cluster.example.id
  data_collection_rule_id = azurerm_monitor_data_collection_rule.example.id

  lifecycle {
    replace_triggered_by = [azurerm_monitor_data_collection_rule.example]
  }

}

[msapitree]

Hi @teowa, thank you for the workaround - which I can confirm is working.

[Joerg-L]

Thanks @teowa for the workaround.