deploying azurerm_linux_function_app results in "Azure Functions runtime is unreachable"

IanMoroney commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.14.6

AzureRM Provider Version

3.0.2

Affected Resource(s)/Data Source(s)

azurerm_linux_function_app

Terraform Configuration Files


terraform {
  required_version = "~> 0.14.6"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0.2"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "function_app" {
  name     = "${var.environment}-${var.name}"
  location = var.location
  tags     = var.tags
}

# tfsec:ignore:azure-storage-default-action-deny tfsec:ignore:azure-storage-queue-services-logging-enabled
resource "azurerm_storage_account" "function_app" {
  # Storage account names must be lower case containing only letter and numbers and less than 24 characters long.
  name                     = "${substr(replace("${var.environment}${var.name}", "-", ""), 0, 20)}func"
  resource_group_name      = azurerm_resource_group.function_app.name
  location                 = azurerm_resource_group.function_app.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  min_tls_version = "TLS1_2"
}

resource "azurerm_service_plan" "function_app" {
  name                = "${var.environment}-${var.name}-plan"
  location            = azurerm_resource_group.function_app.location
  resource_group_name = azurerm_resource_group.function_app.name
  sku_name            = "Y1"
  os_type             = "Linux"

  tags = var.tags
}

resource "azurerm_application_insights" "function_app" {
  name                 = "${var.environment}-${var.name}"
  location             = azurerm_resource_group.function_app.location
  resource_group_name  = azurerm_resource_group.function_app.name
  application_type     = "Node.JS"
  daily_data_cap_in_gb = var.app_insights_daily_data_cap
  retention_in_days    = var.app_insights_retention_by_day
}

# tfsec:ignore:azure-functionapp-enable-http2 tfsec:ignore:azure-functionapp-authentication-enabled
resource "azurerm_linux_function_app" "function_app" {
  name                       = "${var.environment}-${var.name}"
  location                   = azurerm_resource_group.function_app.location
  resource_group_name        = azurerm_resource_group.function_app.name
  service_plan_id            = azurerm_service_plan.function_app.id
  storage_account_name       = azurerm_storage_account.function_app.name
  storage_account_access_key = azurerm_storage_account.function_app.primary_access_key
  https_only                 = true
  builtin_logging_enabled    = false
  identity {
    type = "SystemAssigned"
  }
  site_config {
    http2_enabled                          = false
    application_insights_key               = azurerm_application_insights.function_app.instrumentation_key
    application_insights_connection_string = azurerm_application_insights.function_app.connection_string
  }
  auth_settings {
    enabled = false
  }

  app_settings = {
    FUNCTIONS_WORKER_RUNTIME = "node"
  }

  tags = var.tags

}

variable "environment" {
  default = "dev"
}

variable "name" {
  default = "mybrokenfunction"
}

variable "location" {
  default = "northeurope"
}

variable "tags" {
  type = map(string)
  default = {
    terraform      = "true"
    environment    = "dev"
  }
}

variable "app_insights_daily_data_cap" {
  type        = number
  description = "The daily data cap for the application insights logs."
  default     = "100"
}

variable "app_insights_retention_by_day" {
  type        = number
  description = "The number of days to retain logs in application insights."
  default     = "30"
}

Debug Output/Panic Output

No panic

Expected Behaviour

The azure function app should have deployed, ready for a function to be deployed.

Actual Behaviour

The deployed function app, when looking at the list of Functions shows Azure Functions runtime is unreachable

Deploying a function to the function app via the AZ CLI results in the same error message.

Steps to Reproduce

terraform init
terraform apply

Important Factoids

No response

References

No response

IanMoroney commented 2 years ago

I deleted the function from the portal, and recreated the same function through the portal, then re-ran a plan, and these are all the changes mentioned.

# azurerm_linux_function_app.function_app will be updated in-place
~ resource "azurerm_linux_function_app" "function_app" {
    ~ client_certificate_mode           = "Required" -> "Optional"
    ~ https_only                        = false -> true
      id                                = "/subscriptions/subscriptionid/resourceGroups/dev-mybrokenfunction/providers/Microsoft.Web/sites/dev-mybrokenfunction"
      name                              = "dev-mybrokenfunction"
    ~ tags                              = {
        + "environment"                            = "dev"
        - "hidden-link: /app-insights-resource-id" = "/subscriptions/subscriptionid/resourceGroups/dev-mybrokenfunction/providers/Microsoft.Insights/components/dev-mybrokenfunction" -> null
        + "terraform"                              = "true"
      }
      # (18 unchanged attributes hidden)

    ~ auth_settings {
        ~ token_refresh_extension_hours  = 0 -> 72
          # (4 unchanged attributes hidden)
      }

    + identity {
        + type = "SystemAssigned"
      }

    ~ site_config {
        ~ ftps_state                              = "AllAllowed" -> "Disabled"
        ~ scm_minimum_tls_version                 = "1.0" -> "1.2"
          # (25 unchanged attributes hidden)

        - application_stack {
            - node_version                = "14" -> null
            - use_custom_runtime          = false -> null
            - use_dotnet_isolated_runtime = false -> null
          }

        - cors {
            - allowed_origins     = [
                - "https://portal.azure.com",
              ] -> null
            - support_credentials = false -> null
          }
      }
  }

After applying this change, the function app went back to Azure Functions runtime is unreachable

IanMoroney commented 2 years ago

In order to get the function app to be operable again and resolve the Azure Functions runtime is unreachable error, I had to do the following:

Remove:


  app_settings = {
    FUNCTIONS_WORKER_RUNTIME = "node"
  }

add:

site_config {
    application_stack {
      node_version = "14"
    }

The application_stack block does not appear to be at all optional, and is now in fact required. The application_stack block can not be applied if FUNCTIONS_WORKER_RUNTIME is set.

I feel like the validation and documentation for this needs to be updated to stop people falling into this trap when going through a provider upgrade like I did. I'm surprised (but not surprised) that the Azure SDK lets you deploy a function app in a way which breaks it.

Also, the second validation scenario is if you have the application_stack block with node_version specified, AND you have an app_settings with FUNCTIONS_WORKER_RUNTIME specified, that the provider doesn't block you from doing that, even though the apply always results in:

{"Code":"Conflict","Message":"Parameter with name FUNCTIONS_WORKER_RUNTIME already exists.","Target":null,"Details":[{"Message":"Parameter with name FUNCTIONS_WORKER_RUNTIME already exists."},{"Code":"Conflict"},{"ErrorEntity":{"ExtendedCode":"01013","MessageTemplate":"Parameter with name {0} already exists.","Parameters":["FUNCTIONS_WORKER_RUNTIME"],"Code":"Conflict","Message":"Parameter with name FUNCTIONS_WORKER_RUNTIME already exists."}}],"Innererror":null}

IanMoroney commented 2 years ago

@stephybun , i think this is more than just documentation. There should be some validation preventing you from making this mistake in terraform validate

connershoop commented 2 years ago

This post greatly helped me, thank you @IanMoroney.

After updating from terraforms outdated azurerm_function_app to azurerm_linux_function_app, my function apps continually showed the problem function runtime unreachable. In the configuration it even seemed that azure was interpreting the function_runtime_version of "~4" as a custom value for some reason. Adding custom application stack with dotnet_version = "6.0" solved the problem. Code is below.

application_stack should be required in the documentation until this is fixed.

resource "azurerm_linux_function_app" "name" {
  name                        = var.FUNCTIONAPP_NAME
  location                    = var.LOCATION
  resource_group_name         = azurerm_resource_group.resource_group.name
  service_plan_id             = azurerm_service_plan.functionapps.id
  storage_account_name        = azurerm_storage_account.name.name
  storage_account_access_key  = azurerm_storage_account.name.primary_access_key
  https_only                  = true
  client_certificate_mode     = "Required"
  functions_extension_version = "~4"

  identity {
    type = "SystemAssigned"
  }

  site_config {
    vnet_route_all_enabled   = true
    application_insights_key = azurerm_application_insights.application_insights.instrumentation_key
    http2_enabled            = true

    application_stack {
      dotnet_version = "6.0"
    }
  }
}

xiaxyi commented 1 year ago

@IanMoroney Thanks for raising the issue. I can set the app_setting and the application_stack at the same time without any error:

TF config:

resource "azurerm_linux_function_app" "linuxFunctionApp" {
  name                = "xiaxintest-LFA-node"
  location            = azurerm_resource_group.test.location
  resource_group_name = azurerm_resource_group.test.name
  service_plan_id     = azurerm_service_plan.linuxPlan.id

  storage_account_name       = azurerm_storage_account.test.name
  storage_account_access_key = azurerm_storage_account.test.primary_access_key

app_settings = {
  FUNCTIONS_WORKER_RUNTIME = "node"
}
  site_config {
    application_stack {
      node_version = "14"
    }
  }
}

result:

azurerm_linux_function_app.linuxFunctionApp: Modifications complete after 38s [id=/subscriptions/xxx/resourceGroups/xiaxintestRG-WFA-node/providers/Microsoft.Web/sites/xiaxintest-LFA-node]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

I also checked the code and we don't have any restriction towards the usage of these two properties. Are you using the latest provider?

MattGarnerAWR commented 1 year ago

@xiaxyi,

I believe the problem is it's deploying the runtime as "custom" even if your function app site config is like this:

functions_extension_version = "~4"

and

  site_config {
    application_stack {
      use_custom_runtime = false
    }
  }

In the portal it deploys like this:

When it should be like this:

randompixel commented 1 year ago

In case anyone comes across this issue, I was having these symptoms (Linux Consumption App, Node Azure Functions runtime is unreachable, 503 response for all services) and none of the above suggestions worked.

What fixed it for me was to run a restart after the Terraform and it seems to come back. I'm doing this as a step in the Pipeline after Terraform finished currently. I don't know if it runs a restart as a part of the Terraform or if that could be added?

Also you could try a sync the triggers after updating the code as per here: https://learn.microsoft.com/en-gb/azure/azure-functions/functions-deployment-technologies#trigger-syncing

MarkiEC13 commented 1 year ago

@xiaxyi,

I believe the problem is it's deploying the runtime as "custom" even if your function app site config is like this:

functions_extension_version = "~4"

and
  site_config {
    application_stack {
      use_custom_runtime = false
    }
  }
In the portal it deploys like this:

When it should be like this:

This was the exact issue I was also facing with dotnet-isolated with docker. I had to set 'use_dotnet_isolated_runtime' to 'true' with tf. It still changes runtime back to custom (~4) after a new deployment causing this issue.

hashicorp / terraform-provider-azurerm