search

hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.6k stars 4.64k forks source link

Support for zone_redundant in azurerm_eventhub_namespace in azurerm v4 #27239

Open tw-sematell opened 2 months ago

tw-sematell commented 2 months ago

Is there an existing issue for this?

Community Note

Description

The zone_redundant setting for azurerm_eventhub_namespace was removed in azurerm v4. As the platform engineer, I want to be able to provision zone_redundant Kafka services in Azure without using additional tooling as a workaround.

New or Affected Resource(s)/Data Source(s)

azurerm_eventhub_namespace

Potential Terraform Configuration

resource "azurerm_eventhub_namespace" "example" {
  name                = "evhns-example"
  resource_group_name = azurerm_resource_group.services.name
  location            = azurerm_resource_group.services.location

  zone_redundant = true
  sku            = "Standard"
  capacity       = 1
}

References

No response

rcskosir commented 2 months ago

Linking the PR that removed zone_redundant for context: https://github.com/hashicorp/terraform-provider-azurerm/pull/26611

deese commented 1 month ago

Now when you create a new namespace it is created without the zone redundancy enabled but if you create the namespace from the GUI with the same config, it is enabled based on the location. How should it be done now then?

In the link PR mentions Premium eventhub but in Microsoft documentation states that also Standard has the zone redundancy enabled (and in the GUI works, as mentioned)

nerddtvg commented 1 month ago

Reproduced the issue using the latest AzureRM provider. The Azure Portal forces zone redundancy in zone redundant regions, however it does this by passing the zoneRedundant attribute without the user noticing. And if that property is removed from the ARM template, the Eventhub API creates it with the default false value.

~  terraform --version
Terraform v1.9.6
on darwin_arm64
+ provider registry.terraform.io/hashicorp/azurerm v4.3.0
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>4.0"
    }
  }
}

variable "resource_group_name" {
  type     = string
  default  = "removed"
  nullable = false
}

provider "azurerm" {
  # Configuration options
  features {}

  subscription_id = "removed"
}

resource "azurerm_eventhub_namespace" "this" {
  name                = "eventhubnstestd01"
  resource_group_name = var.resource_group_name
  location            = "westus2"
  sku                 = "Standard"
}
# azurerm_eventhub_namespace.this:
resource "azurerm_eventhub_namespace" "this" {
    auto_inflate_enabled                      = false
    capacity                                  = 1
    dedicated_cluster_id                      = null
    default_primary_connection_string         = (sensitive value)
    default_primary_connection_string_alias   = (sensitive value)
    default_primary_key                       = (sensitive value)
    default_secondary_connection_string       = (sensitive value)
    default_secondary_connection_string_alias = (sensitive value)
    default_secondary_key                     = (sensitive value)
    id                                        = "/subscriptions/removed/resourceGroups/removed/providers/Microsoft.EventHub/namespaces/eventhubnstestd01"
    local_authentication_enabled              = true
    location                                  = "westus2"
    maximum_throughput_units                  = 0
    minimum_tls_version                       = "1.2"
    name                                      = "eventhubnstestd01"
    network_rulesets                          = [
        {
            default_action                 = "Allow"
            ip_rule                        = []
            public_network_access_enabled  = true
            trusted_service_access_enabled = false
            virtual_network_rule           = []
        },
    ]
    public_network_access_enabled             = true
    resource_group_name                       = "removed"
    sku                                       = "Standard"
}
~  az rest -m get -u 'https://management.azure.com/subscriptions/removed/resourceGroups/removed/providers/Microsoft.EventHub/namespaces/eventhubnstestd01?api-version=2024-01-01'
{
  "id": "/subscriptions/removed/resourceGroups/removed/providers/Microsoft.EventHub/namespaces/eventhubnstestd01",
  "location": "westus2",
  "name": "eventhubnstestd01",
  "properties": {
    "createdAt": "2024-10-02T21:29:28.8496231Z",
    "disableLocalAuth": false,
    "geoDataReplication": {
      "locations": [
        {
          "locationName": "westus2",
          "replicaState": "Ready",
          "roleType": "Primary"
        }
      ],
      "maxReplicationLagDurationInSeconds": 0
    },
    "isAutoInflateEnabled": false,
    "kafkaEnabled": true,
    "maximumThroughputUnits": 0,
    "metricId": "removed:eventhubnstestd01",
    "minimumTlsVersion": "1.2",
    "provisioningState": "Succeeded",
    "publicNetworkAccess": "Enabled",
    "serviceBusEndpoint": "[https://eventhubnstestd01.servicebus.windows.net:443/](https://eventhubnstestd01.servicebus.windows.net/)",
    "status": "Active",
    "updatedAt": "2024-10-02T21:29:54.7801714Z",
    "zoneRedundant": false
  },
  "sku": {
    "capacity": 1,
    "name": "Standard",
    "tier": "Standard"
  },
  "tags": {},
  "type": "Microsoft.EventHub/Namespaces"
}

@WodansSon In your original PR you stated this:

The zone_redundant field, per the service team, is now going to be 100% controlled by the API.

However this does not appear to be the case at this time. Is there a timeline for when the API will support this? Or if the direction has been changed, should this property be added back in?

Even looking at the latest preview API (2024-05-01-preview), zoneRedundant is still shown as optional and respected.

nerddtvg commented 1 month ago

I'm going to compound this issue slightly and say that Premium namespaces are impacted twice. The zone_redundant value can be set and is respected by the create action, a Premium namespace is created with zoneRedundant as true. But its value is stored as false because of:

https://github.com/hashicorp/terraform-provider-azurerm/blob/v3.116.0/internal/services/eventhub/eventhub_namespace_resource.go#L361-L364

Because of this, any subsequent plan continually replaces the namespace. So the input should have been ignored completely.

So in versions v4.0+ the zone_redundant property is missing and in earlier versions, it is improperly handled for Premium workspaces.

nerddtvg commented 3 weeks ago

A product group team member directed me to this documentation which they state applies for both Service Bus and Event Hubs: https://learn.microsoft.com/en-us/azure/well-architected/service-guides/service-bus/reliability#configuration-recommendations

Previously it was required to set the property zoneRedundant to true to enable availability zones, however this behavior has changed to enable availability zones by default. Existing namespaces are being migrated to availability zones where possible, and the property zoneRedundant is being deprecated. The property zoneRedundant might still show as false, even when availability zones has been enabled.

No timeline given for the deprecation but at this point we need to correct the provider to prevent redeploying Eventhubs and ignoring the property entirely.

WodansSon commented 2 weeks ago

@nerddtvg, that is exactly right, this field is actually deprecated and is being change for all accounts directly in Azure by the service team. There was some logic added to the Service, to not break Terraform and cause a diff in 3.x providers, which was to echo back whatever the value is that is being sent by Terraform. While the value that is returned from the service might be false the real value in Azure would be true. As of last July the only valid value for the zoneRedundant field is true.

Lobosque commented 2 weeks ago

We cannot upgrade our module to 4.0 because we get an error after removing zone_redundant - following the upgrade guide. internally, The provider is trying to change the property zone_redundant fromtrue to false, but it is immutable.:

│ Error: creating/updating Namespace (Subscription: "xyz"
│ Resource Group Name: "env-performance"
│ Namespace Name: "env-performance-v1"): performing CreateOrUpdate: unexpected status 400 (400 Bad Request) with error: MergeNamespaceFailedWithImmutablePropertyUpdate: ZoneRedundant is an immutable property
│ 
│   with module.service_bus.azurerm_servicebus_namespace.this,
│   on ../service_bus/main.tf line 67, in resource "azurerm_servicebus_namespace" "this":
│   67: resource "azurerm_servicebus_namespace" "this" {
│
WodansSon commented 2 weeks ago

@Lobosque, I will have to check with the service team to see if they removed the shim in the service. It should be dropping the zone_redundant field on the floor, so this should be a no-op.

nerddtvg commented 2 weeks ago

I tested upgrading from 3.116.0 to 4.6.0 with namespaces that had zone_redundant as false and true, both did so without replacement.

############################################
# Testing with zone_redundant = false
############################################

v3.116.0:

  # azurerm_eventhub_namespace.this will be created
  + resource "azurerm_eventhub_namespace" "this" {
      + auto_inflate_enabled                      = false
      + capacity                                  = 1
      + default_primary_connection_string         = (sensitive value)
      + default_primary_connection_string_alias   = (sensitive value)
      + default_primary_key                       = (sensitive value)
      + default_secondary_connection_string       = (sensitive value)
      + default_secondary_connection_string_alias = (sensitive value)
      + default_secondary_key                     = (sensitive value)
      + id                                        = (known after apply)
      + local_authentication_enabled              = true
      + location                                  = "westus2"
      + minimum_tls_version                       = (known after apply)
      + name                                      = "eventhubnstestd01"
      + network_rulesets                          = (known after apply)
      + public_network_access_enabled             = true
      + resource_group_name                       = "rg_name"
      + sku                                       = "Standard"
      + zone_redundant                            = false
    }

############################################

v4.6.0:

azurerm_eventhub_namespace.this: Refreshing state... [id=/subscriptions/subscription/resourceGroups/rg_name/providers/Microsoft.EventHub/namespaces/eventhubnstestd01]

No changes. Your infrastructure matches the configuration.

############################################
# Testing with zone_redundant = true
############################################

v3.116.0:

  # azurerm_eventhub_namespace.this will be created
  + resource "azurerm_eventhub_namespace" "this" {
      + auto_inflate_enabled                      = false
      + capacity                                  = 1
      + default_primary_connection_string         = (sensitive value)
      + default_primary_connection_string_alias   = (sensitive value)
      + default_primary_key                       = (sensitive value)
      + default_secondary_connection_string       = (sensitive value)
      + default_secondary_connection_string_alias = (sensitive value)
      + default_secondary_key                     = (sensitive value)
      + id                                        = (known after apply)
      + local_authentication_enabled              = true
      + location                                  = "westus2"
      + minimum_tls_version                       = (known after apply)
      + name                                      = "eventhubnstestd01"
      + network_rulesets                          = (known after apply)
      + public_network_access_enabled             = true
      + resource_group_name                       = "rg_name"
      + sku                                       = "Standard"
      + zone_redundant                            = true
    }

############################################

v4.6.0:

azurerm_eventhub_namespace.this: Refreshing state... [id=/subscriptions/subscription/resourceGroups/rg_name/providers/Microsoft.EventHub/namespaces/eventhubnstestd01]

No changes. Your infrastructure matches the configuration.
AndersRunningen commented 1 week ago

@nerddtvg From the message above you then mean that this works for azurerm_eventhub_namespace resourses or is it something I have missed?

This when we are trying to apply our Terraform code with terraform version 1.9.6 ad azurerm 4.6.0 we don't really get the update in the plan, but when trying to apply we get the error mentioned in the case.

When looking at the namespace in json view in Azure we can see that the flag is enabled "zoneRedundant": true,, but the provider still tries to disable it.

Have anyone found any workarounds for this?

nerddtvg commented 1 week ago

@AndersRunningen That is correct. I deployed Standard namespaces with zone redundant false and true with 3.116.0. Then I upgraded to 4.6.0, commented out the property, and re-ran the plan.

However, I did not apply 4.6.0 since there was no change. I can test that later today.

AndersRunningen commented 1 week ago

I just tested creating a new servicebus_namespacewith Premium skuand the value I see in the portal afterwards is still "zoneRedundant": false. 

{
    "id": "/subscriptions/3a5fa70a-eee8-45d7-9c55-xxxxxxxxx/resourceGroups/xxxxxxxxx/providers/Microsoft.ServiceBus/namespaces/xxxxxxxxx",
    "name": "xxxxxxxxxx",
    "type": "Microsoft.ServiceBus/Namespaces",
    "location": "westeurope",
    "tags": {
    },
    "properties": {
        "geoDataReplication": {
            "maxReplicationLagDurationInSeconds": 0,
            "locations": [
                {
                    "locationName": "westeurope",
                    "roleType": "Primary",
                    "replicaState": "Ready"
                }
            ]
        },
        "premiumMessagingPartitions": 1,
        "minimumTlsVersion": "1.2",
        "publicNetworkAccess": "Disabled",
        "disableLocalAuth": true,
        "privateEndpointConnections": [
            {
                "id": "/subscriptions/3a5fa70a-eee8-45d7-9c55-xxxxxx/resourceGroups/xxxxxxxxxxx/providers/Microsoft.ServiceBus/namespaces/xxxxxxxxxx/privateEndpointConnections/590fbeb5-7873-4cce-9976-a1f6a1xxxxxx93b5a7",
                "name": "590fbeb5-7873-4cce-9976-a1f6a193xxxxxxxxb5a7",
                "type": "Microsoft.ServiceBus/Namespaces/PrivateEndpointConnections",
                "location": "westeurope",
                "properties": {
                    "provisioningState": "Succeeded",
                    "privateEndpoint": {
                        "id": "/subscriptions/ef0cce31-ae28-4f68-bb2e-xxxxxxxxx/resourceGroups/hub-private-endpoints-rg/providers/Microsoft.Network/privateEndpoints/xxxxxxxxxxxxxxx-private-endpoint"
                    },
                    "privateLinkServiceConnectionState": {
                        "status": "Approved",
                        "description": "Auto-Approved"
                    },
                    "groupIds": [
                        "namespace"
                    ]
                }
            }
        ],
        "zoneRedundant": false,
        "metricId": "3a5fa70a-eee8-45d7-9c55-xxxxxxxx:xxxxxxxxx",
        "serviceBusEndpoint": "https://xxxxxxxxx.servicebus.windows.net:443/",
        "provisioningState": "Succeeded",
        "status": "Active",
        "createdAt": "2024-10-28T11:38:20.6260654Z",
        "updatedAt": "2024-10-28T11:39:18.3083681Z"
    },
    "sku": {
        "name": "Premium",
        "tier": "Premium",
        "capacity": 1
    }
}

Terraform: 1.9.6 AzureRM: 4.6.0

Don't know if I should create a separate issue as we are having issues with a servicebus_namespace and not an eventhub_namespace. Seems like much the same issue though?

zestmf commented 1 week ago

Commenting here as #27497 was closed. Something has definitely changed on the Azure side in the last few weeks as I just retried applying the same Terraform configuration, with no changes in either my code or the version of TF (1.9.6) or AzureRM (4.3.0) vs my earlier post here, and it now applies successfully. The Service Bus resource was modified as expected and there was no sign of the MergeNamespaceFailedWithImmutablePropertyUpdate error I was hitting before. So my problem is resolved, but leaving this info in case it is useful.

empowerScott commented 1 week ago

Commenting here as #27497 was closed. Something has definitely changed on the Azure side in the last few weeks as I just retried applying the same Terraform configuration, with no changes in either my code or the version of TF (1.9.6) or AzureRM (4.3.0) vs my earlier post here, and it now applies successfully. The Service Bus resource was modified as expected and there was no sign of the MergeNamespaceFailedWithImmutablePropertyUpdate error I was hitting before. So my problem is resolved, but leaving this info in case it is useful.

I'm still getting this error:

Error: creating/updating Namespace (Subscription: "" Resource Group Name: "" Namespace Name: "prodsb01"): performing CreateOrUpdate: unexpected status 400 (400 Bad Request) with error: MergeNamespaceFailedWithImmutablePropertyUpdate: ZoneRedundant is an immutable property
with module.servicebus.azurerm_servicebus_namespace.servicebus1-secondary[0]
on ../modules/web/servicebus/main.tf line 36, in resource "azurerm_servicebus_namespace" "servicebus1-secondary"
AndersRunningen commented 4 days ago

@nerddtvg don't know if it is correct to tag you? But do you know if there is any updates here?

nerddtvg commented 4 days ago

@AndersRunningen I'm just an external user like yourself on this. @WodansSon would probably be the best to reply.

I can say what your experiencing is "part of the plan" from our discussions here and my support ticket with MS directly. The false value on newly created resources is possibly correct and incorrect at the same time. The understanding is that newly created resources may be zone redundant but the value is shown as false to prevent conflicts with updates as the default ARM value is false. However, the value could be correct as the migrations to fully zone redundant resources isn't complete and it is really the luck of the draw. And to make matters worse, you have no way of knowing.

alexeyklots commented 4 days ago

Hi, @WodansSon we also have the same issue with our Premium Service Bus which has enabled Zone redundancy. Basically we have a similar problem as described here https://github.com/hashicorp/terraform-provider-azurerm/issues/27239#issuecomment-2427269988

Part of our infrastructure was updated to AzureRM v4.x and part not because of this issue with zone redundancy. Maybe there are some workarounds?

Also @WodansSon maybe we should treat this as a bug and not just enhancement, to speed up the solution if possible.

Thanks for help in any case!

Also tagging @rcskosir