update azure resource "azurerm_lb_backend_address_pool" always destroy and rebuild the "backend address" attribute

MinsonTFW commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

> terraform -v
Terraform v0.13.6
+ provider registry.terraform.io/hashicorp/azurerm v2.46.0

Affected Resource(s)

azurerm_lb_backend_address_pool

Terraform Configuration Files

resource "azurerm_lb_backend_address_pool" "address_pool" {
  name            = "${var.lb_name}-addresspool"
  loadbalancer_id = azurerm_lb.lb.id

  backend_address {
    name                = "f1febd41-b3de-40aa-b377-91f6f9e5795c"
    virtual_network_id  = "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-spoke-sea/providers/Microsoft.Network/virtualNetworks/cloud-gisnonprodsea-vnet-sea"
    ip_address          = "10.236.4.5"
  }

  # dynamic "backend_address" {
  #   for_each = { for x in var.backend_address_settings : x.name => x }
  #   content {
  #     name               = lookup(backend_address.value, "name", null)
  #     virtual_network_id = lookup(backend_address.value, "id", "")
  #     ip_address         = lookup(backend_address.value, "ip", null)
  #   }
  # }
}

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

Debug Output

Terraform will perform the following actions:

  # module.azure_lb.azurerm_lb_backend_address_pool.address_pool will be updated in-place
  ~ resource "azurerm_lb_backend_address_pool" "address_pool" {
        backend_ip_configurations = [
            "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/networkInterfaces/cloud-gisnonprodsea-nic-gis-lin-terraform-azurerm-vm-tests-sea-01/ipConfigurations/ipconfig",
        ]
        id                        = "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/backendAddressPools/cloud-gisnonprodsea-rg-lb01-sea-addresspool"
        load_balancing_rules      = [
            "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/loadBalancingRules/http",
            "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/loadBalancingRules/https",
        ]
        loadbalancer_id           = "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea"
        name                      = "cloud-gisnonprodsea-rg-lb01-sea-addresspool"
        outbound_rules            = []
        resource_group_name       = "cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea"

      - backend_address {
          - name = "f1febd41-b3de-40aa-b377-91f6f9e5795c" -> null
        }
      + backend_address {
          + ip_address         = "10.236.4.5"
          + name               = "f1febd41-b3de-40aa-b377-91f6f9e5795c"
          + virtual_network_id = "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-spoke-sea/providers/Microsoft.Network/virtualNetworks/cloud-gisnonprodsea-vnet-sea"
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Panic Output

Expected Behaviour

There should't be any changes while running terraform plan

Actual Behaviour

The backend_address will be rebuilt

Steps to Reproduce

build a load balancer backend pool without backend address attribute, the configuration:

resource "azurerm_lb_backend_address_pool" "address_pool" {
name            = "${var.lb_name}-addresspool"
loadbalancer_id = azurerm_lb.lb.id
}

using terraform apply to deploy the resource

now the address pool has been built and there is nothing in the backend address pool, so manually add one vm into the pool from azure portal
now the terraform configuration code stay the same and run terraform plan, one change has been shown:
```
  - backend_address {
      - name = "f1febd41-b3de-40aa-b377-91f6f9e5795c" -> null
    }
```
This is expected since we add a new vm into the pool via azure portal.

Now trying to updated the code to match the new changes, add the backend_address to the configuration:

resource "azurerm_lb_backend_address_pool" "address_pool" {
name            = "${var.lb_name}-addresspool"
loadbalancer_id = azurerm_lb.lb.id

backend_address {
name                = "f1febd41-b3de-40aa-b377-91f6f9e5795c"
virtual_network_id  = "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-spoke-sea/providers/Microsoft.Network/virtualNetworks/cloud-gisnonprodsea-vnet-sea"
ip_address          = "10.236.4.5"
}
}

and run terraform plan, however, the output shows that terraform will destroy the backend_address and rebuild it. What we expected here is terraform shows "infrastructure up to date" Also, trying to run terraform refresh and check the tf state file, the "backend address pool" part:

{
  "module": "module.azure_lb",
  "mode": "managed",
  "type": "azurerm_lb_backend_address_pool",
  "name": "address_pool",
  "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
  "instances": [
    {
      "schema_version": 0,
      "attributes": {
        "backend_address": [
          {
            "ip_address": "",
            "name": "f1febd41-b3de-40aa-b377-91f6f9e5795c",
            "virtual_network_id": ""
          }
        ],
        "backend_ip_configurations": [
          "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/networkInterfaces/cloud-gisnonprodsea-nic-gis-lin-terraform-azurerm-vm-tests-sea-01/ipConfigurations/ipconfig"
        ],
        "id": "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/backendAddressPools/cloud-gisnonprodsea-rg-lb01-sea-addresspool",
        "load_balancing_rules": [
          "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/loadBalancingRules/http",
          "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/loadBalancingRules/https"
        ],
        "loadbalancer_id": "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea",
        "name": "cloud-gisnonprodsea-rg-lb01-sea-addresspool",
        "outbound_rules": [],
        "resource_group_name": "cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea",
        "timeouts": null
      },
      "private": "xxx",
      "dependencies": [
        "module.azure_lb.azurerm_lb.lb"
      ]
    }
  ]
},

trying to update the backend address attribute in state file to match the code:

        "backend_address": [
          {
            "ip_address": "10.236.4.5",
            "name": "f1febd41-b3de-40aa-b377-91f6f9e5795c",
            "virtual_network_id": "/subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-spoke-sea/providers/Microsoft.Network/virtualNetworks/cloud-gisnonprodsea-vnet-sea"
          }
        ],

Still, there is unexpected changes while running terraform plan

even we run terraform apply and try to apply the changes, we will still get an error out put:

Error: creating/updating Load Balancer Backend Address Pool "Load Balancer Backend Address Pool: (Backend Address Pool Name \"cloud-gisnonprodsea-rg-lb01-sea-addresspool\" / Load Balancer Name \"cloud-gisnonprodsea-rg-lb01-sea\" / Resource Group \"cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea\")": network.LoadBalancerBackendAddressPoolsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="ModificationOfNICIpConfigBackendPoolNotSupported" Message="Operation on backend pool /subscriptions/xxx/resourceGroups/cloud-gisnonprodsea-rg-terraform-azurerm-vm-tests-sea/providers/Microsoft.Network/loadBalancers/cloud-gisnonprodsea-rg-lb01-sea/backendAddressPools/cloud-gisnonprodsea-rg-lb01-sea-addresspool not allowed since it adds/modifies/deletes backend address pool members that are specified with a network interface IP configuration." Details=[]

on ../main.tf line 39, in resource "azurerm_lb_backend_address_pool" "address_pool":
39: resource "azurerm_lb_backend_address_pool" "address_pool" {

Important Factoids

References

0000

Ksreenivas commented 3 years ago

Hi I have noticed same issue, let us know if it is a bug or the issue on user end itself.

arduinix commented 3 years ago

I also have a similar issue which causes the entire LB to be replaced despite zero configuration change.

dots77 commented 3 years ago

Also having the same issue

gvfnix commented 3 years ago

I am able to reproduce the issue with hashicorp/azurerm=2.46.0. Luckily it can be muted by ignore_changes lifecycle feature

sivasamsivasam commented 3 years ago

I am also having the same issue.

delian commented 3 years ago

Same issue here

shaneguignard commented 2 years ago

I am also having this issue.

muralidar44 commented 2 years ago

1mcreating/updating Load Balancer Backend Address Pool "Load Balancer Backend Address Pool: (Backend Address Pool Name \"mediawikibpappname\" / Load Balancer Name \"applb\" / Resource Group \"mediawikirg\")": network.LoadBalancerBackendAddressPoolsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="ModificationOfNICIpConfigBackendPoolNotSupported" Message="Operation on backend pool /subscriptions/49db0cbb-ac2c-4caa-b82b-39b1426c634d/resourceGroups/mediawikirg/providers/Microsoft.Network/loadBalancers/applb/backendAddressPools/mediawikibpappname not allowed since it adds/modifies/deletes backend address pool members that are specified with a network interface IP configuration." Details=[][0m

I am getting the same error, what is the workaround?

ssulli25 commented 2 years ago

Hello - we are also receiving this issue with "azurerm_lb_backend_address_pool" for both our prod and test environments.

Output below:

╷ │ Error: Provider produced inconsistent result after apply │ │ When applying changes to azurerm_network_interface_backend_address_pool_association.nic_association["vm-nic-agent-02"], provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced an unexpected new value: Root │ │ This is a bug in the provider, which should be reported in the provider's own issue tracker. ╵ ╷ │ Error: updating Loadbalancer "lbi-agent-prod-eus2-azr-001" (resource group "rg-data-prod-agent-azr-01") for Rule "lbi-rule-agent-prod-01": network.LoadBalancersClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidResourceReference" Message="Resource /subscriptions/subscription_id/resourceGroups/rg-data-prod-agent-azr-01/providers/Microsoft.Network/loadBalancers/lbi-agent-prod-eus2-azr-001/probes/lbi-backend-agent-prod-01 referenced by resource /subscriptions/subscription_id/resourceGroups/rg-data-prod-agent-azr-01/providers/Microsoft.Network/loadBalancers/lbi-agent-prod-eus2-azr-001/loadBalancingRules/lbi-rule-agent-prod-01 was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[]
│ │ with azurerm_lb_rule.lb_rule["lb-rule-agent-prod-01"], │ on main.tf line 265, in resource "azurerm_lb_rule" "lb_rule": │ 265: resource "azurerm_lb_rule" "lb_rule" {

Similar to the above commenters, is there an update or workaround here? We've begun to incorporate azure lbs and this is the odd man out.

hemak88 commented 1 year ago

Facing the same issue. Need a fix or workaround

rs-eviden commented 5 months ago

Facing the same issue

hashicorp / terraform-provider-azurerm