Issue with azurerm Terraform Provider - Zone Redundancy Setting

sri-pb commented 11 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.7

AzureRM Provider Version

3.73.0

Affected Resource(s)/Data Source(s)

azurerm_cosmosdb_account

Terraform Configuration Files

resource "azurerm_resource_group" "sample_cosmos_rg" {
  name     = "cosmosdb-sample-rg"
  location = "CanadaCentral"
}

resource "azurerm_cosmosdb_account" "sample_cosmos_account" {
  name                = "cosmosdb-sample-acc"
  location            = azurerm_resource_group.sample_cosmos_rg.location
  resource_group_name = azurerm_resource_group.sample_cosmos_rg.name
  offer_type          = "Standard"
  kind                = "GlobalDocumentDB"
  consistency_policy {
    consistency_level       = "Session"
    max_interval_in_seconds = 5
    max_staleness_prefix    = 100
  }
  enable_automatic_failover = true
  enable_free_tier          = true # Enable the free tier

  geo_location {
    location          = "CanadaCentral"
    failover_priority = 0
    # zone_redundant    = true # Zone redundant configuration
  }

  geo_location {
    location          = "CanadaEast"
    failover_priority = 1
  }

}

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.73.0"
    }
  }
  required_version = "~> 1.5.7"
}

# Configure the Microsoft Azure Provider
provider "azurerm" {
  features {}
}

Debug Output/Panic Output

➜  cosmos-geo-location-az-redundant terraform --version
Terraform v1.5.7
on darwin_arm64
➜  cosmos-geo-location-az-redundant terraform init         

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "~> 3.73.0"...
- Installing hashicorp/azurerm v3.73.0...
- Installed hashicorp/azurerm v3.73.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
➜  cosmos-geo-location-az-redundant terraform providers

Providers required by configuration:
.
└── provider[registry.terraform.io/hashicorp/azurerm] ~> 3.73.0

➜  cosmos-geo-location-az-redundant terraform apply -auto-approve

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_cosmosdb_account.sample_cosmos_account will be created
  + resource "azurerm_cosmosdb_account" "sample_cosmos_account" {
      + access_key_metadata_writes_enabled       = true
      + analytical_storage_enabled               = false
      + connection_strings                       = (sensitive value)
      + create_mode                              = (known after apply)
      + default_identity_type                    = "FirstPartyIdentity"
      + enable_automatic_failover                = true
      + enable_free_tier                         = true
      + enable_multiple_write_locations          = false
      + endpoint                                 = (known after apply)
      + id                                       = (known after apply)
      + is_virtual_network_filter_enabled        = false
      + kind                                     = "GlobalDocumentDB"
      + local_authentication_disabled            = false
      + location                                 = "canadacentral"
      + mongo_server_version                     = (known after apply)
      + name                                     = "cosmosdb-sample-acc"
      + network_acl_bypass_for_azure_services    = false
      + offer_type                               = "Standard"
      + primary_key                              = (sensitive value)
      + primary_readonly_key                     = (sensitive value)
      + primary_readonly_sql_connection_string   = (sensitive value)
      + primary_sql_connection_string            = (sensitive value)
      + public_network_access_enabled            = true
      + read_endpoints                           = (known after apply)
      + resource_group_name                      = "cosmosdb-sample-rg"
      + secondary_key                            = (sensitive value)
      + secondary_readonly_key                   = (sensitive value)
      + secondary_readonly_sql_connection_string = (sensitive value)
      + secondary_sql_connection_string          = (sensitive value)
      + write_endpoints                          = (known after apply)

      + consistency_policy {
          + consistency_level       = "Session"
          + max_interval_in_seconds = (known after apply)
          + max_staleness_prefix    = (known after apply)
        }

      + geo_location {
          + failover_priority = 0
          + id                = (known after apply)
          + location          = "canadacentral"
          + zone_redundant    = false
        }
      + geo_location {
          + failover_priority = 1
          + id                = (known after apply)
          + location          = "canadaeast"
          + zone_redundant    = false
        }
    }

  # azurerm_resource_group.sample_cosmos_rg will be created
  + resource "azurerm_resource_group" "sample_cosmos_rg" {
      + id       = (known after apply)
      + location = "canadacentral"
      + name     = "cosmosdb-sample-rg"
    }

Plan: 2 to add, 0 to change, 0 to destroy.
azurerm_resource_group.sample_cosmos_rg: Creating...
azurerm_resource_group.sample_cosmos_rg: Creation complete after 4s [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg]
azurerm_cosmosdb_account.sample_cosmos_account: Creating...
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [10s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [20s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [30s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [40s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [50s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m0s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m10s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m20s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m30s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m40s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [1m50s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m0s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m10s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m20s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m30s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m40s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [2m50s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m0s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m10s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m20s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m30s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m40s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [3m50s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [4m0s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [4m10s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Still creating... [4m20s elapsed]
azurerm_cosmosdb_account.sample_cosmos_account: Creation complete after 4m26s [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmosdb-sample-acc]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

#############################################################
After uncommenting zone_redundant = true and trying to plan
#############################################################
➜  cosmos-geo-location-az-redundant terraform plan                                            
azurerm_resource_group.sample_cosmos_rg: Refreshing state... [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg]
azurerm_cosmosdb_account.sample_cosmos_account: Refreshing state... [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmosdb-sample-acc]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
➜  cosmos-geo-location-az-redundant terraform init -upgrade

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "~> 3.73.0"...
- Installing hashicorp/azurerm v3.73.0...
- Installed hashicorp/azurerm v3.73.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
➜  cosmos-geo-location-az-redundant terraform plan         
azurerm_resource_group.sample_cosmos_rg: Refreshing state... [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg]
azurerm_cosmosdb_account.sample_cosmos_account: Refreshing state... [id=/subscriptions/<subscriptionsID>/resourceGroups/cosmosdb-sample-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmosdb-sample-acc]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
➜  cosmos-geo-location-az-redundant

Expected Behaviour

When updating an existing "azurerm_cosmosdb_account", terraform (azurerm) provider should detect change in zone_redundant setting and enable/disable zone redundancy accordingly.

Actual Behaviour

The zone_redundant property is being ignored in subsequent changes to an existing "azurerm_cosmosdb_account".

Steps to Reproduce

Run terraform apply (on the Terraform Configuration file provided above)
The terraform apply command should have created a cosmosdb account with ID similar to [id=/subscriptions//resourceGroups/cosmosdb-sample-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmosdb-sample-acc]],
Go to azure portal and validate primary region's availability zone setting (It should be disabled).
Update the Terraform Configuration file provided above by uncommenting the line add zone_redundant = true
Run 'terraform apply' (This command should apply the updated Terraform configuration, including the change to enable zone_redundant. However it doesn't at the moment.)

Note: The issue has been observed across the following versions: Terraform Version: 1.5.7 or 1.5.0 AzureRM Provider Version: 3.31.0 or 3.73.0

Important Factoids

No response

References

No response

henkjan-van-wijk commented 9 months ago

I also see the same behaviour. Using Terraform version: 1.5.7 AzureRM Provider version: 3.74.0

alec-pinson commented 2 months ago

Same issue here AzureRM 3.107.0 (current latest version)

alec-pinson commented 2 months ago

Ok after looking into this abit more it seems that you cannot change zone_redundant after deploying. You would need to destroy, however you can delete/re-add a region with availability zones enabled:-

Add a new region
Failover to this region
Delete the old region
Re-add old region with zone redundancy
Failover

nikhilbhole12 commented 1 month ago

@alec-pinson Are you saying that we will need to setup a process for achieving this via terraform and deployment pipelines? That way we will need to do each step via code to enable zone redundancy.

alec-pinson commented 1 month ago

So if it's brand new cosmos db then setting zone redundancy will work.

If it's already existing then for me the easiest way was manually doing the above steps... if you run a terraform plan after completing the above steps there's no changes.

hashicorp / terraform-provider-azurerm