hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.42k stars 9.5k forks source link

Crash processing the json file for an ECS Task definition. #20363

Closed crysyn closed 5 years ago

crysyn commented 5 years ago

Was asked to resubmit this issue on its own.

Terraform Version v0.11.*, v0.12.0-alpha4

Terraform Configuration Files

resource "aws_ecs_task_definition" "task-definition"
{
    family                  = "${local.project-env-name}"
    container_definitions       = "${file("task-definitions/rabbitmq.json")}"
    tags                        = ["${merge(map("Type", "Task-Definition"), local.common-tags)}"]
}

...
// The JSON file
[
  {
    "name": "first",
    "image": "service-first",
    "cpu": 10,
    "memory": 512,
    "essential": true,
    "portMappings": [
      {
        "containerPort": 80,
        "hostPort": 80
      }
    ]
  },
  {
    "name": "second",
    "image": "service-second",
    "cpu": 10,
    "memory": 256,
    "essential": true,
    "portMappings": [
      {
        "containerPort": 443,
        "hostPort": 443
      }
    ]
  }
]

Crash Output

panic: interface conversion: interface {} is string, not map[string]interface {}
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: goroutine 75 [running]:
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ConfigFieldReader).readMap(0xc00015c300, 0x351aef8, 0x4, 0xc000617b30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/field_reader_config.go:170 +0xed4
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ConfigFieldReader).readField(0xc00015c300, 0xc000061b20, 0x1, 0x1, 0xc00015ce00, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/field_reader_config.go:113 +0xbb7
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ConfigFieldReader).ReadField(0xc00015c300, 0xc000061b20, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0xcbc8b0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/field_reader_config.go:27 +0xe2
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*MultiLevelFieldReader).ReadFieldExact(0xc0004a02e0, 0xc000061b20, 0x1, 0x1, 0x351cbd7, 0x6, 0x0, 0x0, 0x0, 0x0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/field_reader_multi.go:31 +0xfc
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ResourceData).get(0xc0008ac070, 0xc000061b20, 0x1, 0x1, 0xc000061b12, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource_data.go:504 +0x136
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ResourceData).getChange(0xc0008ac070, 0x351aef8, 0x4, 0x3851201, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource_data.go:480 +0x129
2019-02-14T16:21:15.291-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*ResourceData).diffChange(0xc0008ac070, 0x351aef8, 0x4, 0x3519935, 0x1, 0xc0007732f0, 0x5, 0x0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource_data.go:457 +0x9e
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.diffMap(0xc00075fd10, 0x351aef8, 0x4, 0xc000617b30, 0xc000773638, 0x38606a0, 0xc0008ac070, 0x0, 0x0, 0x0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:932 +0x11c
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.diff(0xc00075fd10, 0x351aef8, 0x4, 0xc000617b30, 0xc0004a01e0, 0x38606a0, 0xc0008ac070, 0x0, 0x0, 0x0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:778 +0x531
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.Diff(0xc00075fd10, 0xc000894050, 0xc00008fc20, 0x0, 0x2f48680, 0xc000a5ee00, 0x1, 0x8, 0x4122c8)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:424 +0x20d
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).Diff(0xc00036ce00, 0xc000894050, 0xc00008fc20, 0x2f48680, 0xc000a5ee00, 0x40bc01, 0xc00078bb80, 0x4d13c3)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:250 +0x183
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).Diff(0xc000a6a930, 0xc000894000, 0xc000894050, 0xc00008fc20, 0xc000029180, 0x0, 0x713e0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:296 +0xa3
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/plugin.(*ResourceProviderServer).Diff(0xc000713c80, 0xc0004a00a0, 0xc000060ea0, 0x0, 0x0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/plugin/resource_provider.go:538 +0x5e
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: reflect.Value.call(0xc0004e0b40, 0xc0004a4018, 0x13, 0x351a65c, 0x4, 0xc00078bf18, 0x3, 0x3, 0xc00025aec0, 0x0, ...)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/goenv/versions/1.11.1/src/reflect/value.go:447 +0x450
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: reflect.Value.Call(0xc0004e0b40, 0xc0004a4018, 0x13, 0xc00078bf18, 0x3, 0x3, 0x4, 0x100, 0x0)
2019-02-14T16:21:15.292-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/goenv/versions/1.11.1/src/reflect/value.go:308 +0xab
2019-02-14T16:21:15.293-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: net/rpc.(*service).call(0xc000a7a000, 0xc000424460, 0xc0005f0050, 0xc0005f0060, 0xc00087e680, 0xc0004a0060, 0x29cc280, 0xc0004a00a0, 0x16, 0x29cc2c0, ...)
2019-02-14T16:21:15.293-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/goenv/versions/1.11.1/src/net/rpc/server.go:384 +0x155
2019-02-14T16:21:15.293-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe: created by net/rpc.(*Server).ServeCodec
2019-02-14T16:21:15.293-0800 [DEBUG] plugin.terraform-provider-aws_v1.52.0_x4.exe:      /opt/goenv/versions/1.11.1/src/net/rpc/server.go:481 +0x485
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalDiff, err: unexpected EOF
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalSequence, err: unexpected EOF
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalDiff, err: unexpected EOF
2019/02/14 16:21:15 [TRACE] [walkPlan] Exiting eval tree: aws_ecs_task_definition.hydra-rabbitmq
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalValidateResource, err: Warnings: []. Errors: [unexpected EOF]
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalSequence, err: Warnings: []. Errors: [unexpected EOF]
2019/02/14 16:21:15 [TRACE] [walkPlan] Exiting eval tree: aws_iam_role_policy_attachment.hydra-ecs-ec2-role
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalValidateResource, err: Warnings: []. Errors: [unexpected EOF]
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalSequence, err: Warnings: []. Errors: [unexpected EOF]
2019/02/14 16:21:15 [TRACE] [walkPlan] Exiting eval tree: aws_iam_instance_profile.hydra-ecs-node
2019/02/14 16:21:15 [ERROR] root: eval: *terraform.EvalSequence, err: unexpected EOF
2019/02/14 16:21:15 [TRACE] [walkPlan] Exiting eval tree: aws_vpc.hydra-vpc
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group.hydra-dynamodb"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group.hydra-worker"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_subnet.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_subnet.hydra-datanet"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_lb_target_group.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_subnet.hydra-workers"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group.hydra-rds"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_elasticache_subnet_group.hydra"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_lb.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-to-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_lb_listener.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-from-dynamodb"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-from-rds"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-to-rds"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_db_subnet_group.hydra"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_db_instance.hydra-rds"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group.hydra-elasticache"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_vpc_endpoint.dynamodb"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-to-ec"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-from-ec"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_internet_gateway.hydra"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_vpn_gateway.hydra"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_route_table.hydra-public"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_launch_configuration.hydra-ecs-rabbitmq-nodes"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_default_route_table.hydra-private"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_vpn_connection.hydra-kff"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-to-dynamodb"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_route_table_association.hydra-workers"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_route_table_association.hydra-datanet"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_elasticache_cluster.hydra"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.hydra-worker-from-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_route_table_association.hydra-rabbitmq"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "aws_autoscaling_group.hydra-ecs-rabbitmq-nodes"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "output.summary"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "provider.aws (close)"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "meta.count-boundary (count boundary fixup)"
2019/02/14 16:21:15 [TRACE] dag/walk: upstream errored, not walking "root"
2019/02/14 16:21:15 [DEBUG] plugin: waiting for all plugin processes to complete...
2019-02-14T16:21:15.295-0800 [WARN ] plugin: error closing client during Kill: err="connection is shut down"
2019-02-14T16:21:15.305-0800 [DEBUG] plugin: plugin process exited: path=E:\_projects\hydra-ops\.terraform\plugins\windows_amd64\terraform-provider-aws_v1.52.0_x4.exe

Expected Behavior

I expect the task definition to be processed properly

Actual Behavior

It panics out on the data format, despite being from the documentation.

Steps to Reproduce

Please list the full steps required to reproduce the issue, for example:

  1. terraform init
  2. terraform plan
devonhk commented 5 years ago

Seems to be the same issue I just reported also https://github.com/hashicorp/terraform/issues/20362

crysyn commented 5 years ago

Does indeed, Reported it yesterday under another similar issue and was just reposting it as a root issue

jbardin commented 5 years ago

Hi @crysyn,

Thanks for the crash report. Can you share how you got the cash with a 0.12 build? It looks like the problem is that tags is a map, while the config shown here is a list of maps. That may have done some strange things in 0.11, but 0.12 should catch the error very early on while decoding the config.

This is the output from master:

Error: Incorrect attribute value type

  on main.tf line 10, in resource "aws_ecs_task_definition" "task-definition":
  10:   tags                  = ["${merge(map("Type", "Task-Definition"), local.common-tags)}"]

Inappropriate value for attribute "tags": map of string required.
crysyn commented 5 years ago

Wow, how did I miss this was in the tags and not the json file..

In any case, here is how the local.common-tags is built up

{
    common-tags = "${map(
        "Owner",        "${var.owner-name}",
        "Project",      "${var.project-name}",
        "Environment",  "${var.environment-name}",
        "Name",         "${local.project-env-name}"
    )}"
    project-env-name    = "${var.project-name}-${var.environment-name}"
}

This is a tag block I use extensively through out the files to keep everything created targeted at the project they are a part of.. but now that I am thinking back over this there is another area I removed the tags from due to errors being reported for that specific resource. If I can find that again I will include that object as well.

crysyn commented 5 years ago

Oh, @jbardin, all I did was download a copy of the v0.12.0-alpha4 and run it from a local folder after wiping out the state (which is what I did in most of my tests between the versions). Sorry I forgot to include this before.

Changing the tags block out to

    tags                        =
    {
        Owner           = "${var.owner-name}"
        Project         = "${var.project-name}"
        Environment     = "${var.environment-name}"
        Name            = "${local.project-env-name}"
    }

did get it to accept the tag block if that helps as well.

jbardin commented 5 years ago

Sorry, I'm not sure I'm following here. The issue I mentioned isn't with the content of the tags map, but rather that your original config is assigning a list containing a map to tags. This would be caught right away in 0.12 and should never get to that panic. In 0.11 the list-of-maps data can sneak through the limited type system, and will produce that panic you've shown.

If you have a config that panics under 0.12, I can verify it against master for you, but I think the underlying issue here is likely solved.

crysyn commented 5 years ago

I would say its covered by the error output on master then. it Panic'd on the alpha4.

jbardin commented 5 years ago

OK, I'll mark this as resolved then, since it will no longer be possible to feed incorrect data types into the legacy sdk.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.