azurerm_function_app_slot: Provider produced inconsistent final plan

robinkb commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.26

provider.azurerm v2.14.0
provider.random v2.2.1

Affected Resource(s)

azurerm_function_app_slot

Terraform Configuration Files

resource "azurerm_function_app" "middleware_api" {
  name                = "func-${var.customer}-${var.env}-middleware-api"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  app_service_plan_id = azurerm_app_service_plan.middleware_api.id

  storage_account_name       = azurerm_storage_account.middleware_api.name
  storage_account_access_key = azurerm_storage_account.middleware_api.primary_access_key

  version = "~3"

  identity {
    type = "UserAssigned"

    identity_ids = [
      azurerm_user_assigned_identity.documents_storage_account_manage.id,
      azurerm_user_assigned_identity.keyvault_read.id
    ]
  }

  app_settings = {
    # Required settings for Function Apps to work.
    APPINSIGHTS_INSTRUMENTATIONKEY = azurerm_application_insights.middleware_api.instrumentation_key
    FUNCTIONS_WORKER_RUNTIME       = "node"
    WEBSITE_NODE_DEFAULT_VERSION   = "~10"
    WEBSITE_RUN_FROM_PACKAGE       = "1"
    # Something else is setting this, and not setting it here triggers changes to remove it.
    SCM_DO_BUILD_DURING_DEPLOYMENT = "false"

    # Custom app warm-up settings.
    WEBSITE_SWAP_WARMUP_PING_PATH     = "/api/health-check"
    WEBSITE_SWAP_WARMUP_PING_STATUSES = "200"
    # Azure recommends always setting this except for WCF apps.
    WEBSITE_ADD_SITENAME_BINDINGS_IN_APPHOST_CONFIG = "1"

    KEY_VAULT_URI = azurerm_key_vault.secrets.vault_uri

    # Application settings.
    COSMOSDB_DBNAME   = azurerm_cosmosdb_sql_database.****_sync.name
    COSMOSDB_HOST     = "https://${azurerm_cosmosdb_account.cosmos_db.name}.documents.azure.com"
    COSMOSDB_PASSWORD = azurerm_cosmosdb_account.cosmos_db.primary_master_key
    COSMOSDB_PORT     = "443"
    COSMOSDB_USER     = azurerm_cosmosdb_account.cosmos_db.name
    ****_API_TOKEN    = data.azurerm_key_vault_secret.****_api_token.value
    ****_BASE_PATH    = var.****_base_path
  }
}

resource "azurerm_function_app_slot" "middleware_api_staging" {
  name                = "staging"
  location            = azurerm_function_app.middleware_api.location
  resource_group_name = azurerm_function_app.middleware_api.resource_group_name

  app_service_plan_id = azurerm_function_app.middleware_api.app_service_plan_id

  storage_account_name       = azurerm_function_app.middleware_api.storage_account_name
  storage_account_access_key = azurerm_function_app.middleware_api.storage_account_access_key

  version = azurerm_function_app.middleware_api.version

  dynamic "identity" {
    for_each = azurerm_function_app.middleware_api.identity

    content {
      type         = identity.value.type
      identity_ids = identity.value.identity_ids
    }
  }

  function_app_name = azurerm_function_app.middleware_api.name

  # We might need to support different settings on the staging slot, but for now,
  # this is by far the easiest way to keep the settings in sync.
  app_settings = azurerm_function_app.middleware_api.app_settings
}

Debug Output

Error: Provider produced inconsistent final plan

When expanding the plan for azurerm_function_app_slot.middleware_api_staging
to include new values learned so far during apply, provider
"registry.terraform.io/-/azurerm" produced an invalid new value for
.identity[0].identity_ids[0]: was
cty.StringVal("/subscriptions/****/resourcegroups/DEV/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-****-dev-storage-account-manage"),
but now
cty.StringVal("/subscriptions/****/resourcegroups/DEV/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-****-dev-keyvault-read").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

(Is the true debug output necessary for this issue?)

Expected Behavior

Terraform applies the changes without error.

Actual Behavior

Terraform errors out.

Steps to Reproduce

terraform apply

Important Factoids

I have other function app slots in my configuration, but no other has multiple managed identities, and none of them have the dynamic snippet in them. Maybe the dynamic snippet is causing the issue?

References

N/A

neil-yechenwei commented 4 years ago

@robinkb, thanks for opening this issue. Seems azurerm_function_app is updated and the property identity_ids is changed before updating azurerm_function_app_slot at second tf apply so that terraform found identity_ids is different with the value at first tf apply and threw above error message.

robinkb commented 4 years ago

I think that the issue is that the identity_ids field does not have a consistent order. Terraform keeps showing changes in that field during the plan phase.

# azurerm_function_app.middleware_api will be updated in-place
~ resource "azurerm_function_app" "middleware_api" {

    # Snip...

    ~ identity {
        ~ identity_ids = [
            - "/subscriptions/****/resourcegroups/DEV/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-****-dev-keyvault-read",
            "/subscriptions/****/resourcegroups/DEV/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-****-dev-storage-account-manage",
            + "/subscriptions/****/resourcegroups/DEV/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-****-dev-keyvault-read",
        ]
        type         = "UserAssigned"
    }

    # Snip...
}

Matthewsre commented 3 years ago

I am having the same problem for both azurerm_function_app and azurerm_app_service .

azurerm_app_service:

  ~ resource "azurerm_app_service" "microservice" {

        identity {
          ~ identity_ids = [
              - "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-sql-dev",
                "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-keyvault-dev",
              + "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-sql-dev",
            ]
            # (3 unchanged attributes hidden)
        }

azurerm_function_app:

  ~ resource "azurerm_function_app" "microservice" {
      ~ identity {
          ~ identity_ids = [
              - "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-sql-dev",
                "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-cosmos-dev",
                # (1 unchanged element hidden)
                "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-servicebus-dev",
              + "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-sql-dev",
            ]
            # (3 unchanged attributes hidden)
        }

I was going to try working around this by looking up the order using the data source for azurerm_app_service and data source for aazurerm_function_app but the identity is not available on the results of azurerm_app_service to determine the order.

@neil-yechenwei any work arounds you can suggest? I have valid use cases where these change so am unable to use ignore_changes. This solution also has several azurerm_app_service and azurerm_function_app across multiple regions which is also inflating this and causing large diff results.

hashicorp / terraform-provider-azurerm