Failure on terraform apply for cloudflare_ruleset

nickbabkin commented 3 years ago

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version

Terraform v1.0.6 on darwin_amd64

provider registry.terraform.io/cloudflare/cloudflare v3.1.0
provider registry.terraform.io/hashicorp/aws v3.60.0
provider registry.terraform.io/hashicorp/http v2.1.0
provider registry.terraform.io/hashicorp/null v3.1.0
provider registry.terraform.io/tchupp/env v0.0.2

Affected resource(s)

cloudflare_ruleset

Terraform configuration files

example.com.yaml:
zone_example_com_rulesets: 
 - name: "Managed WAF"
   description: "Cloudflare Managed WAF ruleset"
   kind: "zone"
   phase: "http_request_firewall_managed"
   rules:
   - action: "execute"
     enabled: true
     action_parameters:
     - id: "efb7b8c949ac4650a09736fc376e9aee"
       overrides: 
         - categories: []
           rules: []
       uri: []
     expression: "true"
     description: "Execute Cloudflare Managed Ruleset on my zone-level phase entry point ruleset"

variables.tf:
# Rulesets
variable "rulesets" {
  description = "Provides a Cloudflare firewall rulesets."
  type = list(object({
    name                       = string           # (Required) Name of the ruleset.
    description                = string           # (Required) Brief summary of the ruleset and its intended use.
    kind                       = string           # (Required) Type of Ruleset to create. Valid values are "custom", "managed", "root", "schema" or "zone".
    phase                      = string           # (Required) Point in the request/response lifecycle where the ruleset will be created. Valid values are "ddos_l4", "ddos_l7", "http_request_firewall_custom", "http_request_firewall_managed", "http_request_late_transform", "http_request_main", "http_request_sanitize", "http_request_transform", "http_response_firewall_managed", "magic_transit", or "http_ratelimit".
    shareable_entitlement_name = optional(string) # (Optional) Name of entitlement that is shareable between entities.
    rules = list(object({                         # (Required) List of rules to apply to the ruleset
      enabled = optional(bool)                    # (Optional) Whether the rule is active.
      description = optional(string)              # (Optional) Brief summary of the ruleset rule and its intended use.
      action = string                             # (Required) Action to perform in the ruleset rule. Valid values are "block", "challenge", "ddos_dynamic", "execute", "force_connection_close", "js_challenge", "log", "rewrite", "score", or "skip".
      expression = string                         # (Required) Criteria for an HTTP request to trigger the ruleset rule action. Uses the Firewall Rules expression language based on Wireshark display filters. Refer to the Firewall Rules language documentation for all available fields, operators, and functions.
      action_parameters = optional(list(object({  # (Required) List of parameters that configure the behavior of the ruleset rule action
        id        = optional(string)              # (Optional) Identifier of the action parameter to modify.
        increment = optional(string)              # (Optional)
        overrides = optional(list(object({        # (Optional) List of override configurations to apply to the ruleset
          enabled = optional(bool)                # (Optional) Defines if the current ruleset-level override enables or disables the ruleset.
          categories = optional(list(object({     # (Optional) List of tag-based overrides
            category = optional(string)           # (Optional) Tag name to apply the ruleset rule override to.
            action   = optional(string)           # (Optional) Action to perform in the tag-level override. Valid values are "block", "challenge", "ddos_dynamic", "execute", "force_connection_close", "js_challenge", "log", "rewrite", "score", or "skip".
            enabled  = optional(bool)             # (Optional) Defines if the current tag-level override enables or disables the ruleset rules with the specified tag.
          })))                                    #
          rules = optional(list(object({          # (Optional) List of rule-based overrides
            id              = optional(string)    # (Optional) Rule ID to apply the override to.
            action          = optional(string)    # (Optional) Action to perform in the rule-level override. Valid values are "block", "challenge", "ddos_dynamic", "execute", "force_connection_close", "js_challenge", "log", "rewrite", "score", or "skip".
            enabled         = optional(bool)      # (Optional) Defines if the current rule-level override enables or disables the rule.
            score_threshold = optional(string)    # (Optional) Anomaly score threshold to apply in the ruleset rule override. Only applicable to modsecurity-based rulesets.
          })))                                    #
        })))                                      #
        products = optional(list(string))         # (Optional) Products to target with the actions. Valid values are "bic", "hot", "ratelimit", "securityLevel", "uablock", "waf" or "zonelockdown".
        ruleset  = optional(string)               # (Optional) Which ruleset to target. Valid value is "current".
        uri = optional(list(object({              # (Optional) List of URI properties to configure for the ruleset rule when performing URL rewrite transformations
          path = optional(list(object({           # (Optional) URI path configuration when performing a URL rewrite
            expression = optional(string)         # (Optional) Expression that defines the updated (dynamic) value of the URI path or query string component. Conflicts with `value`.
            query      = optional(string)         # (Optional) Static string value of the updated URI path or query string component. Conflicts with `expression`.
          })))                                    #
          query = optional(list(object({          # (Optional) Query string configuration when performing a URL rewrite
            expression = optional(string)         # (Optional) Expression that defines the updated (dynamic) value of the URI path or query string component. Conflicts with `value`.
            query      = optional(string)         # (Optional) Static string value of the updated URI path or query string component. Conflicts with `expression`.
          })))
        })))
        version = optional(string) # (Optional)
      })))
    }))
  }))
  default = []
}

main.tf:
locals {
  rulesets         = var.enabled ? { for k, rule in var.rulesets : k => rule } : {}
}

// Firewall rulesets
resource "cloudflare_ruleset" "default" {
  for_each = local.rulesets

  zone_id = cloudflare_zone.default.*.id[0]

  name                       = each.value.name
  description                = each.value.description
  kind                       = each.value.kind
  phase                      = each.value.phase
  shareable_entitlement_name = each.value.shareable_entitlement_name

  dynamic "rules" {
    for_each = each.value.rules
    content {
      action = rules.value.action
      dynamic "action_parameters" {
        for_each = rules.value.action_parameters
        content {
          id        = action_parameters.value.id
          increment = action_parameters.value.increment
          dynamic "overrides" {
            for_each = action_parameters.value.overrides
            content {
              enabled = overrides.value.enabled
              dynamic "categories" {
                for_each = overrides.value.categories
                content {
                  category = categories.value.category
                  action   = categories.value.action
                  enabled  = categories.value.enabled
                }
              }
              dynamic "rules" {
                for_each = overrides.value.rules
                content {
                  id              = rules.value.id
                  action          = rules.value.action
                  enabled         = rules.value.enabled
                  score_threshold = rules.value.score_threshold
                }
              }
            }
          }
          products = action_parameters.value.products
          ruleset  = action_parameters.value.ruleset
          dynamic "uri" {
            for_each = action_parameters.value.uri
            content {
              dynamic "path" {
                for_each = uri.value.path
                content {
                  value = path.value.value
                }
              }
              dynamic "query" {
                for_each = uri.value.query
                content {
                  value = query.value.value
                }
              }
            }
          }
          version = action_parameters.value.version
        }
      }

      expression  = rules.value.expression
      description = rules.value.description
      enabled     = rules.value.enabled
    }
  }
}

Debug output

module.zone_example_com.cloudflare_ruleset.default["0"]: Creating... 2021-09-29T17:46:50.302Z [INFO] Starting apply for module.zone_example_com.cloudflare_ruleset.default["0"] 2021-09-29T17:46:50.303Z [DEBUG] module.zone_example_com.cloudflare_ruleset.default["0"]: applying the planned Create change 2021-09-29T17:46:50.304Z [INFO] provider.terraform-provider-cloudflare_v3.1.0: 2021/09/29 17:46:50 [DEBUG] unknown key encountered in buildRulesetRulesFromResource for action parameters: products: timestamp=2021-09-29T17:46:50.304Z 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: PANIC: interface conversion: interface {} is nil, not map[string]interface {} 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: goroutine 55 [running]: 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/cloudflare/terraform-provider-cloudflare/cloudflare.buildRulesetRulesFromResource({0xc00063e240, 0x1d}, {0xe40580, 0xc0006d14b8}) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_ruleset.go:665 +0x1ce7 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/cloudflare/terraform-provider-cloudflare/cloudflare.resourceCloudflareRulesetCreate(0x0, {0xfdb3c0, 0xc000146300}) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_ruleset.go:341 +0x2d7 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).create(0xe9eb40, {0x1206248, 0xc000588100}, 0x2, {0xfdb3c0, 0xc000146300}) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:318 +0x178 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).Apply(0xc0001dce00, {0x1206248, 0xc000588100}, 0xc00063c3f0, 0xc0009ba560, {0xfdb3c0, 0xc000146300}) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:456 +0x871 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(GRPCProviderServer).ApplyResourceChange(0xc0005900d8, {0x1206248, 0xc000588100}, 0xc00062c910) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/grpc_provider.go:955 +0x9aa 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(server).ApplyResourceChange(0xc000535ac0, {0x12062f0, 0xc0009e2000}, 0x0) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-go@v0.3.1/tfprotov5/server/server.go:332 +0x6c 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0xf9dbc0, 0xc000535ac0}, {0x12062f0, 0xc0009e2000}, 0xc000388360, 0x0) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: github.com/hashicorp/terraform-plugin-go@v0.3.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:380 +0x170 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc.(Server).processUnaryRPC(0xc0006dba40, {0x1217c28, 0xc0002ae600}, 0xc000bb3560, 0xc00023e2a0, 0x18b0e60, 0x0) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc@v1.40.0/server.go:1297 +0xccf 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc.(Server).handleStream(0xc0006dba40, {0x1217c28, 0xc0002ae600}, 0xc000bb3560, 0x0) 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc@v1.40.0/server.go:1626 +0xa2a 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc.(Server).serveStreams.func1.2() 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc@v1.40.0/server.go:941 +0x98 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: created by google.golang.org/grpc.(Server).serveStreams.func1 2021-09-29T17:46:50.306Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: google.golang.org/grpc@v1.40.0/server.go:939 +0x294 2021-09-29T17:46:50.308Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-09-29T17:46:50.308Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.1.0/linux_amd64/terraform-provider-cloudflare_v3.1.0 pid=241 error="exit status 2" 2021-09-29T17:46:50.308Z [ERROR] plugin.(*GRPCProvider).ApplyResourceChange: error="rpc error: code = Unavailable desc = transport is closing"

Panic output

Stack trace from the terraform-provider-cloudflare_v3.1.0 plugin:

panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 55 [running]: github.com/cloudflare/terraform-provider-cloudflare/cloudflare.buildRulesetRulesFromResource({0xc00063e240, 0x1d}, {0xe40580, 0xc0006d14b8}) github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_ruleset.go:665 +0x1ce7 github.com/cloudflare/terraform-provider-cloudflare/cloudflare.resourceCloudflareRulesetCreate(0x0, {0xfdb3c0, 0xc000146300}) github.com/cloudflare/terraform-provider-cloudflare/cloudflare/resource_cloudflare_ruleset.go:341 +0x2d7 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).create(0xe9eb40, {0x1206248, 0xc000588100}, 0x2, {0xfdb3c0, 0xc000146300}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:318 +0x178 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).Apply(0xc0001dce00, {0x1206248, 0xc000588100}, 0xc00063c3f0, 0xc0009ba560, {0xfdb3c0, 0xc000146300}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/resource.go:456 +0x871 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(GRPCProviderServer).ApplyResourceChange(0xc0005900d8, {0x1206248, 0xc000588100}, 0xc00062c910) github.com/hashicorp/terraform-plugin-sdk/v2@v2.7.1/helper/schema/grpc_provider.go:955 +0x9aa github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(server).ApplyResourceChange(0xc000535ac0, {0x12062f0, 0xc0009e2000}, 0x0) github.com/hashicorp/terraform-plugin-go@v0.3.1/tfprotov5/server/server.go:332 +0x6c github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0xf9dbc0, 0xc000535ac0}, {0x12062f0, 0xc0009e2000}, 0xc000388360, 0x0) github.com/hashicorp/terraform-plugin-go@v0.3.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:380 +0x170 google.golang.org/grpc.(Server).processUnaryRPC(0xc0006dba40, {0x1217c28, 0xc0002ae600}, 0xc000bb3560, 0xc00023e2a0, 0x18b0e60, 0x0) google.golang.org/grpc@v1.40.0/server.go:1297 +0xccf google.golang.org/grpc.(Server).handleStream(0xc0006dba40, {0x1217c28, 0xc0002ae600}, 0xc000bb3560, 0x0) google.golang.org/grpc@v1.40.0/server.go:1626 +0xa2a google.golang.org/grpc.(Server).serveStreams.func1.2() google.golang.org/grpc@v1.40.0/server.go:941 +0x98 created by google.golang.org/grpc.(Server).serveStreams.func1 google.golang.org/grpc@v1.40.0/server.go:939 +0x294

Error: The terraform-provider-cloudflare_v3.1.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.

╷ │ Error: Plugin did not respond │ │ with module.zone_example_com.cloudflare_ruleset.default["0"], │ on modules/zone/main.tf line 248, in resource "cloudflare_ruleset" "default": │ 248: resource "cloudflare_ruleset" "default" { │ │ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details. ╵ 2021-09-29T17:46:50.956Z [DEBUG] provider: plugin exited

Expected output

Resource added

Actual output

Panic

Steps to reproduce

Use the configuration above
Terraform plan (succeeds)
Terraform apply (fails)

Additional factoids

No response

References

No response

nickbabkin commented 3 years ago

The issue is hidden somewhere here:

2021-09-29T18:20:24.666Z [WARN] Provider "registry.terraform.io/cloudflare/cloudflare" produced an invalid plan for module.zone_example_com.cloudflare_ruleset.default["0"], but we are tolerating it because it is using the legacy plugin SDK. The following problems may be the cause of any confusing errors from downstream operations:

.rules[0].action_parameters[0].products: planned value cty.NullVal(cty.Set(cty.String)) does not match config value cty.SetValEmpty(cty.String) module.zone_example_com.cloudflare_ruleset.default["0"]: Creating... 2021-09-29T18:20:24.667Z [INFO] Starting apply for module.zone_example_com.cloudflare_ruleset.default["0"] 2021-09-29T18:20:24.667Z [DEBUG] module.zone_example_com.cloudflare_ruleset.default["0"]: applying the planned Create change 2021-09-29T18:20:24.669Z [INFO] provider.terraform-provider-cloudflare_v3.1.0: 2021/09/29 18:20:24 [DEBUG] unknown key encountered in buildRulesetRulesFromResource for action parameters: products: timestamp=2021-09-29T18:20:24.669Z 2021-09-29T18:20:24.671Z [DEBUG] provider.terraform-provider-cloudflare_v3.1.0: PANIC: interface conversion: interface {} is nil, not map[string]interface {}`

jacobbednarz commented 3 years ago

can you please provide a simple reproduction case for this? i don't have the bandwidth to delve into this elaborate module to diagnose the issue. i suspect you're setting empty fields that shouldn't be set though.

nickbabkin commented 3 years ago

@jacobbednarz I'll try to provide a simple reproduction.

A short summary: According to the logs, clearly something is wrong with action_parameters.products value. But as you see in the config, I'm not even providing it. This is super strange.

panic: interface conversion: interface {} is nil, not map[string]interface {}

cytopia commented 3 years ago

FYI: I've linked an issue for cloudflare_firewall_rule which also seems to have an issue with the parameter products

jacobbednarz commented 3 years ago

@nickbabkin i think there is a bit of confusion here as the issue isn't with the products attribute, that is just a log line saying it doesn't have a mapping associated which is fine. the backtrace shows your issue is that L665 which is related to categories and you are applying them as an empty list.

jacobbednarz commented 3 years ago

closing this off as we haven't managed to get a minimal reproduction test case. feel free to let me know if you get a test case and we can dig further into it.

cloudflare / terraform-provider-cloudflare