azurerm_function_app is not allowing AzureWebJobsStorage app setting to pull from Azure Key Vault

BradAF commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.13.3

provider registry.terraform.io/hashicorp/azuread v1.0.0
provider registry.terraform.io/hashicorp/azurerm v2.31.1

Affected Resource(s)

azurerm_function_app

Terraform Configuration Files

resource "azurerm_function_app" "func01" {
  name                       = local.function_app_name
  location                   = azurerm_resource_group.all.location
  resource_group_name        = azurerm_resource_group.all.name
  app_service_plan_id        = azurerm_app_service_plan.plan01.id
  storage_account_name       = azurerm_storage_account.sa01.name
  storage_account_access_key = azurerm_storage_account.sa01.primary_access_key
  https_only                 = true # Pulled from ARM template
  client_affinity_enabled    = true # Pulled from ARM template
  enabled                    = true # Pulled from ARM template
  version                    = "~2" # Per Request

  site_config {
    always_on                   = false # Pulled from ARM template
    ftps_state                  = "AllAllowed" #Pulled from ARM template
    http2_enabled               = false # Pulled from ARM template
    scm_use_main_ip_restriction = false # Pulled from ARM template; switched to false on feedback from Christoph
#   scm_type                    = "None" # Pulled from ARM template; defaults to none, explicit causes issues
    use_32_bit_worker_process   = true # Pulled from ARM template
    websockets_enabled          = false # Pulled from ARM template
    min_tls_version             = "1.2" # Pulled from ARM template

    cors {
      support_credentials = true # Pulled from ARM template

      allowed_origins     = [
        "https://functions.azure.com",
        "https://functions-staging.azure.com",
        "https://functions-next.azure.com",
        "http://localhost:4200"
      ] # Pulled from ARM template
    }

    dynamic "ip_restriction" {
      for_each = local.ip_allow_list
        content {
          ip_address = ip_restriction.value
          priority = ip_restriction.key + 1
        }
    } # Pulled from ARM template

  }

  auth_settings {
    enabled = true
    token_store_enabled = true
    default_provider = "AzureActiveDirectory"
    issuer = "https://sts.windows.net/${data.azurerm_subscription.current.tenant_id}/" # Manually set

    active_directory {
      client_id = var.client_id
      client_secret = data.azurerm_key_vault_secret.clientsecret.value
      allowed_audiences = ["https://${local.function_app_name}.azurewebsites.net/.auth/login/aad/callback"] # Manually set
    }

    #Allows the function app to retrieve a token for Graph API.
    additional_login_params = {
      "response_type"="code id_token",
      "resource"="00000003-0000-0000-c000-000000000000"
    }

  }

  identity {
    type = "SystemAssigned"
  }

  app_settings = {
    AzureWebJobsStorage                   = "@Microsoft.KeyVault(VaultName=${data.azurerm_key_vault.akv0.name};SecretName=${azurerm_key_vault_secret.azurewebjobsstorage.name};SecretVersion=)"
    NextIterationLogicAppAddress          = "@Microsoft.KeyVault(VaultName=${data.azurerm_key_vault.akv0.name};SecretName=${azurerm_key_vault_secret.nextiterationlogicappaddress.name};SecretVersion=)"
    WEB_HOST                              = "@Microsoft.KeyVault(VaultName=${data.azurerm_key_vault.akv0.name};SecretName=${azurerm_key_vault_secret.webhost.name};SecretVersion=)"
    AdminRoleKey                          = "Admin"
    ApproverRoleKey                       = "Approver"
    productLines                          = "<redacted>"
    sas-time-range                        = "from:-5 minutes to:+10 minutes"
    SkipNextIterationSteps                = "false"
    APPINSIGHTS_INSTRUMENTATIONKEY        = azurerm_application_insights.ai01.instrumentation_key
    APPLICATIONINSIGHTS_CONNECTION_STRING = azurerm_application_insights.ai01.connection_string
    STORAGEACCOUNTSAS                     = data.azurerm_storage_account_sas.sas01.sas
  }

}

Expected Behavior

The AzureWebJobsStorage app setting should be configured to use the Azure Key Vault value, and be labeled as a 'Key Vault Reference'
Terraform should not try to overwrite the configuration unless it is changed.

Actual Behavior

The AzureWebJobsStorage app setting is configured locally, as an 'App Config' instead of 'Key Vault Reference'
Every subsequent deployment attempts to overwrite the AzureWebJobsStorage app setting with the correct Azure Key Vault setting.

Steps to Reproduce

terraform apply

Important Factoids

Other app settings configured to use the Azure Key Vault work as expected.

References

#6909

nmanzi commented 3 years ago

Is that the correct syntax for key vault references? I'm using the following and it's working "fine"

@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.name.id})

*fine isn't really fine - there's a separate issue where changes to the secret values changes the URI and causes an "inconsistent final plan" error when setting the function app config.

BradAF commented 3 years ago

I think so, I don’t recall where I had originally found it in the Microsoft Docs, I’ll have to check my notes.

The other two app settings (NextIterationLogicAppAddreas and WEB_HOST) use the same syntax and are working as expected, though. I forgot to mention, the App Config that it shows for AzureWebJobsStorage is correct, but it contains a key which is why we would prefer that it show only the KeyVault connection and pull from there. I think if syntax were incorrect, it would just show the incorrect string which makes me think that the function app is overwriting it after it has been set by Terraform.

nmanzi commented 3 years ago

Indeed, that syntax is fine according to https://docs.microsoft.com/en-us/azure/app-service/app-service-key-vault-references

You may be right that the AzureWebJobsStorage is being overwritten. If you reveal the hidden value, is it the full config string or the key vault reference?

BradAF commented 3 years ago

Yup, under the hidden value the full config string appears; not the key vault reference or any string similar to what it would be if I had typo'd it:

joaocpribeiro commented 3 years ago

If the function app is in consumption or premium plan, also the app setting WEBSITE_CONTENTAZUREFILECONNECTIONSTRING is automatically created with the storage account connection string as app config. It would also be nice to allow this app setting to be a key vault reference. This app setting is not even recognized in the state, it is marked as new on any apply. I am writing in the issue because I guess it is a change that can be made together.

alainlecluse commented 3 years ago

is this still a bug? or is this an unintentional feature? The documentation clearly states: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/function_app#app_settings

The values for AzureWebJobsStorage and FUNCTIONS_EXTENSION_VERSION will be filled by other input arguments and shouldn't be configured separately. AzureWebJobsStorage is filled based on storage_account_name and storage_account_access_key. FUNCTIONS_EXTENSION_VERSION is filled based on version.

still hope that this will be changed.

antonyrajendran commented 3 years ago

Hope this can be change to use KeyVault Reference, as this is exposing the storage connection string in the Application Settings.

lonegunmanb commented 2 years ago

An update: AzureWebJobsStorage related support may be imported in new resources azurerm_linux_function_app and azurerm_windows_function_app, which were not released yet, rather than in current resource type.

jmcee1 commented 2 years ago

This actually works with storage_connection_string (deprecated) argument. Problem is with setting storage_account_name and storage_account_access_key. App settings get replaced as stated in documentation and storage_account_access_key doesn't support @Microsoft.KeyVault() syntax. This works like a charm (apart from deprecation warning): storage_connection_string = @Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.name.versionless_id})

kiranpradeep commented 2 years ago

An update: AzureWebJobsStorage related support may be imported in new resources azurerm_linux_function_app and azurerm_windows_function_app, which were not released yet, rather than in current resource type.

@lonegunmanb But, can azurerm_windows_function_app/azurerm_linux_function_app be used for consumption plan functions? As per documentation, a requirement argument is service_plan_id which is of an app service plan.

lonegunmanb commented 2 years ago

service_plan_id

Hi @kiranpradeep ,

I think a new issue would be better to ask for support for azurerm_windows_function_app/azurerm_linux_function_app consumption plan.

kiranpradeep commented 2 years ago

I think a new issue would be better to ask for support for azurerm_windows_function_app/azurerm_linux_function_app consumption plan.

In my view, I was more pointing out that the comment made by @lonegunmanb on 13/Oct/2021, which suggested azurerm_windows_function_app/azurerm_linux_function_app as a workaround for this issue is not matching with the documentation. Maybe I am wrong.

Eitherway, I followed that suggestion and had raised new issue at #15627. I raised in this thread itself so that others who follow this thread and see that suggestion, could save some hours by not following it for now.

xiaxyi commented 2 years ago

@BradAF Can you check if the property AzureWebJobsSecretStorageType was correctly set?

A Blob Storage SAS URL for a second storage account used for key storage. By default, Functions uses the account set in AzureWebJobsStorage. When using this secret storage option, make sure that AzureWebJobsSecretStorageType isn't explicitly set or is set to blob. To learn more, see Secret repositories.

hashicorp / terraform-provider-azurerm