Error provisioning App Service after 20s

AlexandreSi commented 4 years ago

Hi, I am having trouble provisioning an app service. The service is created but Terraform crashes and doesn't store the newly created resource in the state.

Terraform (and AzureRM Provider) Version

Terraform v0.12.18
+ provider.azuread v0.7.0
+ provider.azurerm v1.39.0

Affected Resource(s)

azurerm_app_service

Terraform Configuration Files

resource "azurerm_container_registry" "acr" {
  name                     = "${terraform.workspace}ContainerRegistry"
  resource_group_name      = azurerm_resource_group.project.name
  location                 = azurerm_resource_group.project.location
  sku                      = "Basic"
  admin_enabled            = true
}

resource "azurerm_app_service_plan" "api" {
  name                     = "api"
  resource_group_name      = azurerm_resource_group.project.name
  location                 = azurerm_resource_group.project.location
  kind                     = "Linux"

  # Mandatory for Linux plans
  reserved                 = true
  sku {
    tier                   = "PremiumV2"
    size                   = "P1v2"
  }
}

resource "azurerm_app_service" "api" {
  name                     = "api-${terraform.workspace}"
  resource_group_name      = azurerm_resource_group.project.name
  location                 = azurerm_resource_group.project.location
  app_service_plan_id      = azurerm_app_service_plan.api.id

  site_config {
    always_on              = true
    linux_fx_version       = "DOCKER|${azurerm_container_registry.acr.login_server}/project-api:latest"
  }

  app_settings = {
    // settings for app service
    WEBSITES_ENABLE_APP_SERVICE_STORAGE = false
    WEBSITES_PORT = 5000
    SSH_PORT = 2222
    DOCKER_REGISTRY_SERVER_URL = "https://${azurerm_container_registry.acr.login_server}"
    DOCKER_REGISTRY_SERVER_USERNAME = azurerm_container_registry.acr.admin_username
    DOCKER_REGISTRY_SERVER_PASSWORD = azurerm_container_registry.acr.admin_password
    // env variables for the application
    DATABASE_NAME = var.database_name
    DATABASE_USER = "${var.database_user}@${azurerm_postgresql_server.db.name}"
    DATABASE_PASSWORD = var.database_password
    DATABASE_HOST = azurerm_postgresql_server.db.fqdn
    DATABASE_PORT = 5432
    APP_SECRET_KEY = var.api_secret_key[terraform.workspace]
    APP_ALLOWED_HOSTS = "*"
    LOG_LEVEL = "INFO"
    USE_S3 = true
    AWS_ACCESS_KEY_ID = var.aws_access_key_id_django
    AWS_SECRET_ACCESS_KEY = var.aws_secret_access_key_django
    AWS_STORAGE_BUCKET_NAME = "${var.static_storage_bucket_name_prefix}-${terraform.workspace}"
    BASE_URL = "https://${var.api-base-url[terraform.workspace]}"
  }
}

Debug Output

2020/01/06 14:40:44 [DEBUG] azurerm_app_service.api: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalMaybeTainted
2020/01/06 14:40:44 [TRACE] EvalMaybeTainted: azurerm_app_service.api encountered an error during creation, so it is now marked as tainted
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalWriteState
2020/01/06 14:40:44 [TRACE] EvalWriteState: removing state object for azurerm_app_service.api
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalApplyProvisioners
2020/01/06 14:40:44 [TRACE] EvalApplyProvisioners: azurerm_app_service.api has no state, so skipping provisioners
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalMaybeTainted
2020/01/06 14:40:44 [TRACE] EvalMaybeTainted: azurerm_app_service.api encountered an error during creation, so it is now marked as tainted
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalWriteState
2020/01/06 14:40:44 [TRACE] EvalWriteState: removing state object for azurerm_app_service.api
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalIf
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalIf
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalWriteDiff
2020/01/06 14:40:44 [TRACE] <root>: eval: *terraform.EvalApplyPost
2020/01/06 14:40:44 [ERROR] <root>: eval: *terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing
2020/01/06 14:40:44 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2020/01/06 14:40:44 [TRACE] [walkApply] Exiting eval tree: azurerm_app_service.api
2020/01/06 14:40:44 [TRACE] vertex "azurerm_app_service.api": visit complete
2020/01/06 14:40:44 [TRACE] dag/walk: upstream of "provider.azurerm (close)" errored, so skipping
2020/01/06 14:40:44 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2020/01/06 14:40:44 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020-01-06T14:40:44.216+0100 [DEBUG] plugin: plugin exited

Panic Output

panic: interface conversion: interface {} is nil, not map[string]interface {}

Expected Behavior

Terraform apply exits normally.

Actual Behavior

Crash

Steps to Reproduce

terraform apply

Important Factoids

No

tombuildsstuff commented 4 years ago

hi @AlexandreSi

Thanks for opening this issue.

So that we can take a look into this would you mind pasting the full stack trace (e.g. the ~50 lines below panic: interface conversion: interface {} is nil, not map[string]interface {} here?

Thanks!

AlexandreSi commented 4 years ago

Hi @tombuildsstuff,

Thanks for your anwser, here is the crash log after the panic:

[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: panic: interface conversion: interface {} is nil, not map[string]interface {}
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: goroutine 73 [running]:
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/azurerm/helpers/azure.ExpandAppServiceLogs(0x427d6a0, 0xc00077e920, 0x4, 0x427d6a0, 0xc00077e920, 0x1)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/azurerm/helpers/azure/app_service.go:1262 +0xca9
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/azurerm.resourceArmAppServiceCreate(0xc0004bdab0, 0x439ebc0, 0xc000610200, 0x0, 0x0)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/azurerm/resource_arm_app_service.go:299 +0x14f3
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).Apply(0xc000290d00, 0xc0000baaa0, 0xc000c273e0, 0x439ebc0, 0xc000610200, 0xc000c8db01, 0xc00099e730, 0x4386a80)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/resource.go:305 +0x3b4
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Apply(0xc000842700, 0xc000c3da68, 0xc0000baaa0, 0xc000c273e0, 0xc00099e228, 0xc00000ef40, 0x4388be0)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/provider.go:294 +0x18f
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc00000ea88, 0x507cc60, 0xc000ab05a0, 0xc00008ecc0, 0xc00000ea88, 0xc000ab05a0, 0xc000a3dbd0)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin/grpc_provider.go:885 +0x884
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x4706aa0, 0xc00000ea88, 0x507cc60, 0xc000ab05a0, 0xc00008ec60, 0x0, 0x507cc60, 0xc000ab05a0, 0xc00102a000, 0xf87)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5/tfplugin5.pb.go:3189 +0x23e
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003d0420, 0x50d1080, 0xc0004e2a80, 0xc001026000, 0xc00076acc0, 0x8b72660, 0x0, 0x0, 0x0)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:995 +0x466
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc.(*Server).handleStream(0xc0003d0420, 0x50d1080, 0xc0004e2a80, 0xc001026000, 0x0)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1275 +0xda6
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000586100, 0xc0003d0420, 0x50d1080, 0xc0004e2a80, 0xc001026000)
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:710 +0x9f
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4: created by github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
[DEBUG] plugin.terraform-provider-azurerm_v1.39.0_x4:   /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:708 +0xa1
[DEBUG] plugin: plugin process exited: path=/Users/alexandre/projects/liet/terraform-azure/all/.terraform/plugins/darwin_amd64/terraform-provider-azurerm_v1.39.0_x4 pid=85684 error="exit status 2"
[DEBUG] azurerm_app_service.api: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
[TRACE] <root>: eval: *terraform.EvalMaybeTainted
[TRACE] EvalMaybeTainted: azurerm_app_service.api encountered an error during creation, so it is now marked as tainted
[TRACE] <root>: eval: *terraform.EvalWriteState
[TRACE] EvalWriteState: removing state object for azurerm_app_service.api
[TRACE] <root>: eval: *terraform.EvalApplyProvisioners
[TRACE] EvalApplyProvisioners: azurerm_app_service.api has no state, so skipping provisioners
[TRACE] <root>: eval: *terraform.EvalMaybeTainted
[TRACE] EvalMaybeTainted: azurerm_app_service.api encountered an error during creation, so it is now marked as tainted
[TRACE] <root>: eval: *terraform.EvalWriteState
[TRACE] EvalWriteState: removing state object for azurerm_app_service.api
[TRACE] <root>: eval: *terraform.EvalIf
[TRACE] <root>: eval: *terraform.EvalIf
[TRACE] <root>: eval: *terraform.EvalWriteDiff
[TRACE] <root>: eval: *terraform.EvalApplyPost
[ERROR] <root>: eval: *terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing
[ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
[TRACE] [walkApply] Exiting eval tree: azurerm_app_service.api
[TRACE] vertex "azurerm_app_service.api": visit complete
[TRACE] dag/walk: upstream of "provider.azurerm (close)" errored, so skipping
[TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
[TRACE] dag/walk: upstream of "root" errored, so skipping
[DEBUG] plugin: plugin exited

AlexandreSi commented 4 years ago

Hi @tombuildsstuff, Do you see an obvious error in the stack trace or do you think it might need some further investigation and more time? Thanks,

AlexandreSi commented 4 years ago

Still present in version 1.40

ghost commented 4 years ago

This has been released in version 1.42.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.42.0"
}
# ... other configuration ...

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

hashicorp / terraform-provider-azurerm