search

terraform-linters / tflint-ruleset-aws

TFLint ruleset for terraform-provider-aws
Mozilla Public License 2.0
327 stars 71 forks source link

aws_resource_missing_tags appears broken #48

Closed cdobbyn closed 3 years ago

cdobbyn commented 3 years ago

I noticed there was an existing ticket about this rule, but I don't think they're related (I could be wrong).

After updating to tflint v0.23.1 and installing this module at v0.1.2 I am now seeing a large number of errors coming out of the aws_resource_missing_tags rule.

I have a rule created like this:

rule "aws_resource_missing_tags" {
  enabled = true
  tags = ["Environment", "Owner", "Terraform", "Name", "Type"]
  exclude = []
}

We are configuring tags using a merge operation with a variable that gets merged in:

variable "tags" {
  description = "A map of tags to use"
  type        = map(string)
  default     = {}
}
locals {
  tags = merge({
    Environment = "unknown"
    Owner       = "unknown"
    }, var.tags, {
    Type      = "cdn/cloudfront"
    Terraform = "true"
  })
}

This appears to fail this check with the following types of errors:

Error: Failed to check `aws_resource_missing_tags` rule: gob: type not registered for interface: map[string]string

Error: Failed to check `aws_resource_missing_tags` rule: gob: type not registered for interface: map[string]string

2021-01-17T02:43:33.850-0800 [WARN]  plugin: error closing client during Kill: err="unexpected EOF"
2021-01-17T02:43:33.850-0800 [WARN]  plugin: plugin failed to exit gracefully
2021-01-17T02:43:31.979-0800 [WARN]  plugin: error closing client during Kill: err="unexpected EOF"
2021-01-17T02:43:31.979-0800 [WARN]  plugin: plugin failed to exit gracefully

Error: Failed to check `aws_resource_missing_tags` rule: gob: type not registered for interface: []rules.awsAutoscalingGroupTag

2021-01-17T02:43:33.018-0800 [WARN]  plugin: error closing client during Kill: err="unexpected EOF"
2021-01-17T02:43:33.018-0800 [WARN]  plugin: plugin failed to exit gracefully

Maybe I'm doing something wrong, but I hope someone can either point me in the right direction or maybe I found a bug?

wata727 commented 3 years ago

Hmmm, I couldn't reproduce the problem. Maybe I think the cause is Autoscaling group. Would you please share the minimum code that would cause the problem?

cdobbyn commented 3 years ago

The code for the asg is pretty boring since it's all just using variables. Looks like the way my colleague split the tags out is a little weird. Could be the problem, but both ways are documented as valid (multiple tag blocks and tags variable). It looks like I can reproduce the one error:

Error: Failed to check `aws_resource_missing_tags` rule: gob: type not registered for interface: []rules.awsAutoscalingGroupTag

Example:

locals {
  tags = merge(
    {
      Owner       = "unspecified"
      Environment = "unspecified"
    },
    var.tags,
    {
      Type      = format("broker/rabbitmq")
      Terraform = "true"
    }
  )

  tags_asg_format = null_resource.tags_as_list_of_maps.*.triggers
}

variable "tags" {
  type        = map(string)
  default     = {}
}

resource "null_resource" "tags_as_list_of_maps" {
  count = length(keys(local.tags))

  triggers = {
    "key"                 = keys(local.tags)[count.index]
    "value"               = values(local.tags)[count.index]
    "propagate_at_launch" = "true"
  }
}

resource "aws_autoscaling_group" "this" {
  name_prefix          = local.asg_name
  launch_configuration = aws_launch_configuration.this.*.name[0]
  vpc_zone_identifier  = var.subnets
  max_size             = var.asg_max_size
  min_size             = var.asg_min_size
  desired_capacity     = var.asg_desired_capacity

  load_balancers            = var.load_balancers
  health_check_grace_period = var.health_check_grace_period
  health_check_type         = var.health_check_type

  min_elb_capacity          = var.min_elb_capacity
  wait_for_elb_capacity     = var.wait_for_elb_capacity
  target_group_arns         = [aws_lb_target_group.this.arn, aws_lb_target_group.this_tls.arn]
  default_cooldown          = var.default_cooldown
  force_delete              = var.force_delete
  termination_policies      = var.termination_policies
  suspended_processes       = var.suspended_processes
  placement_group           = var.placement_group
  enabled_metrics           = var.enabled_metrics
  metrics_granularity       = var.metrics_granularity
  wait_for_capacity_timeout = var.wait_for_capacity_timeout
  protect_from_scale_in     = var.protect_from_scale_in

  tags = local.tags_asg_format
}
cdobbyn commented 3 years ago

If I modify the asg code to use "tag" this seems to fail with the same error as well:

resource "aws_autoscaling_group" "this" {

  name_prefix          = local.asg_name
  launch_configuration = aws_launch_configuration.this.*.name[0]
  vpc_zone_identifier  = var.subnets
  max_size             = var.asg_max_size
  min_size             = var.asg_min_size
  desired_capacity     = var.asg_desired_capacity

  load_balancers            = var.load_balancers
  health_check_grace_period = var.health_check_grace_period
  health_check_type         = var.health_check_type

  min_elb_capacity          = var.min_elb_capacity
  wait_for_elb_capacity     = var.wait_for_elb_capacity
  target_group_arns         = [aws_lb_target_group.this.arn, aws_lb_target_group.this_tls.arn]
  default_cooldown          = var.default_cooldown
  force_delete              = var.force_delete
  termination_policies      = var.termination_policies
  suspended_processes       = var.suspended_processes
  placement_group           = var.placement_group
  enabled_metrics           = var.enabled_metrics
  metrics_granularity       = var.metrics_granularity
  wait_for_capacity_timeout = var.wait_for_capacity_timeout
  protect_from_scale_in     = var.protect_from_scale_in

//  tags = local.tags_asg_format

  dynamic "tag" {
    for_each = local.tags
    content {
      key                 = tag.key
      value               = tag.value
      propagate_at_launch = "true"
    }
  }
}
cdobbyn commented 3 years ago

Here's the debug log. I also tried statically assigning the tags variable and tag blocks and the error still occurred.

23:11:04 config.go:96: [INFO] Load config: ../../../.tflint.hcl
23:11:04 config.go:311: [DEBUG] Config loaded
23:11:04 config.go:312: [DEBUG]   Module: false
23:11:04 config.go:313: [DEBUG]   Force: false
23:11:04 config.go:314: [DEBUG]   IgnoreModules: map[string]bool{}
23:11:04 config.go:315: [DEBUG]   Varfiles: []string{}
23:11:04 config.go:316: [DEBUG]   Variables: []string{}
23:11:04 config.go:317: [DEBUG]   DisabledByDefault: false
23:11:04 config.go:318: [DEBUG]   Rules: map[string]*tflint.RuleConfig{"aws_db_instance_invalid_type":(*tflint.RuleConfig)(0xc0008e9350), "aws_elasticache_cluster_invalid_type":(*tflint.RuleConfig)(0xc0008e9380), "aws_resource_missing_tags":(*tflint.RuleConfig)(0xc0008e9290), "aws_route_not_specified_target":(*tflint.RuleConfig)(0xc0008e92f0), "aws_route_specified_multiple_targets":(*tflint.RuleConfig)(0xc0008e9320), "aws_s3_bucket_name":(*tflint.RuleConfig)(0xc0008e92c0), "terraform_deprecated_interpolation":(*tflint.RuleConfig)(0xc0008e9080), "terraform_documented_outputs":(*tflint.RuleConfig)(0xc0008e90e0), "terraform_documented_variables":(*tflint.RuleConfig)(0xc0008e9110), "terraform_module_pinned_source":(*tflint.RuleConfig)(0xc0008e9170), "terraform_naming_convention":(*tflint.RuleConfig)(0xc0008e91a0), "terraform_required_providers":(*tflint.RuleConfig)(0xc0008e9200), "terraform_required_version":(*tflint.RuleConfig)(0xc0008e9230), "terraform_standard_module_structure":(*tflint.RuleConfig)(0xc0008e9260), "terraform_typed_variables":(*tflint.RuleConfig)(0xc0008e9140), "terraform_unused_declarations":(*tflint.RuleConfig)(0xc0008e90b0), "terraform_workspace_remote":(*tflint.RuleConfig)(0xc0008e91d0)}
23:11:04 config.go:319: [DEBUG]   Plugins: map[string]*tflint.PluginConfig{"aws":(*tflint.PluginConfig)(0xc0008e93b0)}
23:11:04 option.go:48: [DEBUG] CLI Options
23:11:04 option.go:49: [DEBUG]   Module: false
23:11:04 option.go:50: [DEBUG]   Force: false
23:11:04 option.go:51: [DEBUG]   IgnoreModules: map[string]bool{}
23:11:04 option.go:52: [DEBUG]   EnableRules: []string(nil)
23:11:04 option.go:53: [DEBUG]   DisableRules: []string(nil)
23:11:04 option.go:54: [DEBUG]   Only: []string(nil)
23:11:04 option.go:55: [DEBUG]   Varfiles: []string{}
23:11:04 option.go:56: [DEBUG]   Variables: []string{}
23:11:04 loader.go:57: [INFO] Initialize new loader
23:11:04 loader.go:82: [INFO] Load configurations under .
23:11:04 loader.go:90: [INFO] Module inspection is disabled. Building a root module without children...
23:11:04 loader.go:170: [INFO] Load values files
23:11:04 runner.go:50: [INFO] Initialize new runner for root
23:11:04 discovery.go:68: [INFO] Plugin `aws` found
2021-01-24T23:11:04.422-0800 [DEBUG] plugin: starting plugin: path=/home/cdobbyn/.tflint.d/plugins/tflint-ruleset-aws args=[/home/cdobbyn/.tflint.d/plugins/tflint-ruleset-aws]
2021-01-24T23:11:04.422-0800 [DEBUG] plugin: plugin started: path=/home/cdobbyn/.tflint.d/plugins/tflint-ruleset-aws pid=337225
2021-01-24T23:11:04.422-0800 [DEBUG] plugin: waiting for RPC address: path=/home/cdobbyn/.tflint.d/plugins/tflint-ruleset-aws
2021-01-24T23:11:04.484-0800 [DEBUG] plugin: using plugin: version=7
2021-01-24T23:11:04.484-0800 [DEBUG] plugin.tflint-ruleset-aws: plugin address: address=/tmp/plugin015940228 network=unix timestamp=2021-01-24T23:11:04.484-0800
23:11:04 provider.go:62: [INFO] Prepare rules
23:11:04 provider.go:74: [DEBUG] `terraform_deprecated_interpolation` is enabled
23:11:04 provider.go:74: [DEBUG] `terraform_documented_outputs` is enabled
23:11:04 provider.go:74: [DEBUG] `terraform_documented_variables` is enabled
23:11:04 provider.go:74: [DEBUG] `terraform_module_pinned_source` is enabled
23:11:04 provider.go:74: [DEBUG] `terraform_naming_convention` is enabled
23:11:04 provider.go:76: [DEBUG] `terraform_standard_module_structure` is disabled
23:11:04 provider.go:74: [DEBUG] `terraform_typed_variables` is enabled
23:11:04 provider.go:76: [DEBUG] `terraform_required_version` is disabled
23:11:04 provider.go:76: [DEBUG] `terraform_required_providers` is disabled
23:11:04 provider.go:74: [DEBUG] `terraform_workspace_remote` is enabled
23:11:04 provider.go:74: [DEBUG] `terraform_unused_declarations` is enabled
23:11:04 provider.go:90: [INFO]   8 rules enabled
23:11:04 terraform_unused_declaration.go:113: [DEBUG] Cannot find references in expression, ignoring: Invalid reference: A reference to a resource type must be followed by at least one attribute access, specifying the resource name.
2021-01-24T23:11:04.762-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:04 [DEBUG] Accessing to the `aws` provider config in the root module
2021-01-24T23:11:04.763-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:04 [INFO] Initialize AWS Client
2021-01-24T23:11:04.764-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:04 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2021-01-24T23:11:04.766-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:04 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2021-01-24T23:11:05.244-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_db_instance.*.parameter_group_name` attribute
2021-01-24T23:11:05.245-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_db_instance.*.instance_class` attribute
2021-01-24T23:11:05.245-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_db_instance.*.instance_class` attribute
2021-01-24T23:11:05.245-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_dynamodb_table.*.stream_view_type` attribute
2021-01-24T23:11:05.246-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_elasticache_cluster.*.parameter_group_name` attribute
2021-01-24T23:11:05.246-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_elasticache_cluster.*.node_type` attribute
2021-01-24T23:11:05.247-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_elasticache_cluster.*.node_type` attribute
2021-01-24T23:11:05.247-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_instance.*.instance_type` attribute
2021-01-24T23:11:05.247-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_mq_broker.*.engine_type` attribute
2021-01-24T23:11:05.248-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_mq_configuration.*.engine_type` attribute
2021-01-24T23:11:05.250-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_accessanalyzer_analyzer` resource
2021-01-24T23:11:05.252-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_acm_certificate` resource
2021-01-24T23:11:05.252-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_acmpca_certificate_authority` resource
2021-01-24T23:11:05.252-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_alb` resource
2021-01-24T23:11:05.253-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_alb_target_group` resource
2021-01-24T23:11:05.253-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_ami` resource
2021-01-24T23:11:05.254-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_ami_copy` resource
2021-01-24T23:11:05.254-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_ami_from_instance` resource
2021-01-24T23:11:05.255-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_api_key` resource
2021-01-24T23:11:05.255-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_client_certificate` resource
2021-01-24T23:11:05.255-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_domain_name` resource
2021-01-24T23:11:05.256-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_rest_api` resource
2021-01-24T23:11:05.256-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_stage` resource
2021-01-24T23:11:05.256-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_usage_plan` resource
2021-01-24T23:11:05.257-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_api_gateway_vpc_link` resource
2021-01-24T23:11:05.257-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_apigatewayv2_api` resource
2021-01-24T23:11:05.258-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_apigatewayv2_domain_name` resource
2021-01-24T23:11:05.258-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_apigatewayv2_stage` resource
2021-01-24T23:11:05.258-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_apigatewayv2_vpc_link` resource
2021-01-24T23:11:05.259-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_gateway_route` resource
2021-01-24T23:11:05.259-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_mesh` resource
2021-01-24T23:11:05.259-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_route` resource
2021-01-24T23:11:05.260-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_virtual_gateway` resource
2021-01-24T23:11:05.260-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_virtual_node` resource
2021-01-24T23:11:05.260-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_virtual_router` resource
2021-01-24T23:11:05.261-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appmesh_virtual_service` resource
2021-01-24T23:11:05.261-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_appsync_graphql_api` resource
2021-01-24T23:11:05.261-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_athena_workgroup` resource
2021-01-24T23:11:05.261-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_autoscaling_group` resource
2021-01-24T23:11:05.263-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [DEBUG] Walk `aws_autoscaling_group.*.tag` block
Failed to check ruleset. An error occurred:

Error: Failed to check `aws_resource_missing_tags` rule: gob: type not registered for interface: []rules.awsAutoscalingGroupTag

2021-01-24T23:11:05.268-0800 [DEBUG] plugin.tflint-ruleset-aws: 2021/01/24 23:11:05 [ERR] plugin: plugin server: accept unix /tmp/plugin015940228: use of closed network connection
2021-01-24T23:11:05.272-0800 [DEBUG] plugin: plugin process exited: path=/home/cdobbyn/.tflint.d/plugins/tflint-ruleset-aws pid=337225
2021-01-24T23:11:05.273-0800 [DEBUG] plugin: plugin exited
cdobbyn commented 3 years ago

So here's the static blocks I tried:

  tag {
    key = "Name"
    value = "asdf"
    propagate_at_launch = "true"
  }
  tags = [
    {
      key = "Name"
      value = "asdf"
      propogate_at_launch = "true"
    }
  ]

For these I had simplified my tflint config to only specify this:

rule "aws_resource_missing_tags" {
  enabled = true
//  tags = ["Environment", "Owner", "Terraform", "Name", "Type"]
  tags = ["Name"]
  exclude = []
}
wata727 commented 3 years ago

Thank you for sharing examples. Understood.

This issue is caused by the cty-based expression evaluation. https://github.com/terraform-linters/tflint-ruleset-aws/blob/3ab728b59252c8c91268404cb88c739858e74f35/rules/aws_resource_missing_tags.go#L229-L230

Previously, it works but the plugin system doesn't support mapping evaluation results to structs. I will consider how to fix it.