Using secrets input results in "Invalid function argument" error on map_secrets

syphernl commented 3 years ago

Describe the Bug

After upgrading to v0.51.0 a Terraform plan now shows the following error for each of the defined secrets in the secrets input:

Error: Invalid function argument
  on .terraform/modules/container/main.tf line 20, in locals:
  20:   secrets_values      = var.map_secrets != null ? values(var.map_secrets) : [for m in local.secrets_vars : lookup(m, "value")]
Invalid value for "inputMap" parameter: the given object has no attribute
"value".

No map_secrets input is defined in the module invocation
The secrets contains a number of references to SSM parameters (which worked fine prior to v0.51.0).

Expected Behavior

No error.

Steps to Reproduce

Create a container definition such as:

module "demo_container" {
  source          = "git::https://github.com/cloudposse/terraform-aws-ecs-container-definition.git?ref=0.49.2"
  container_name  = "demo-container"
  container_image = "nginxdemos/hello:latest"

  secrets = [
    # App specific
    {
      name      = "APP_KEY",
      valueFrom = module.store_write_demo.arn_map[local.demo_app_key_parameter]
    }
  ]

  environment = [
    {
      name  = "APP_ENV",
      value = "production",
    }
  ]

  port_mappings = [
    {
      containerPort = 80
      hostPort      = 80
      protocol      = "tcp"
    }
  ]

  log_configuration = {
    logDriver = "awslogs"
    options = {
      "awslogs-region"        = var.aws_region
      "awslogs-group"         = join("", aws_cloudwatch_log_group.our_log_group.*.name)
      "awslogs-stream-prefix" = "app"
    }
  }

  healthcheck = {
    command     = ["CMD-SHELL", "curl -f http://localhost:80/health || exit 1"]
    retries     = 5
    timeout     = 5
    interval    = 30
    startPeriod = 30
  }
}

Screenshots

If applicable, add screenshots or logs to help explain your problem.

Environment:

Anything that will help us triage the bug will help. Here are some ideas:

OS: Alpine Linux 3.13
Terraform: 0.14.7
AWS Provider: 3.29.1

Additional Context

PR with changes: #120 & #123

syphernl commented 3 years ago

@davidvasandani Would you mind taking look at this one please?

nitrocode commented 3 years ago

First of all, thanks for creating this issue!

I see the issue. It's using value instead of valueFrom key. The PR https://github.com/cloudposse/terraform-aws-ecs-container-definition/pull/126 should resolve it.

module "demo_container" {
  source          = "git::https://github.com/cloudposse/terraform-aws-ecs-container-definition.git?ref=0.52.0"
  container_name  = "demo-container"
  container_image = "nginxdemos/hello:latest"

  secrets = [
    # App specific
    {
      name      = "APP_KEY",
      valueFrom = "test"
    }
  ]

  environment = [
    {
      name  = "APP_ENV",
      value = "production",
    }
  ]

  port_mappings = [
    {
      containerPort = 80
      hostPort      = 80
      protocol      = "tcp"
    }
  ]

  healthcheck = {
    command     = ["CMD-SHELL", "curl -f http://localhost:80/health || exit 1"]
    retries     = 5
    timeout     = 5
    interval    = 30
    startPeriod = 30
  }
}

output "demo_container" {
  value = module.demo_container.json_map_encoded
}

$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

demo_container = "{\"cpu\":0,\"environment\":[{\"name\":\"APP_ENV\",\"value\":\"production\"}],\"essential\":true,\"healthCheck\":{\"command\":[\"CMD-SHELL\",\"curl -f http://localhost:80/health || exit 1\"],\"interval\":30,\"retries\":5,\"startPeriod\":30,\"timeout\":5},\"image\":\"nginxdemos/hello:latest\",\"mountPoints\":[],\"name\":\"demo-container\",\"portMappings\":[{\"containerPort\":80,\"hostPort\":80,\"protocol\":\"tcp\"}],\"readonlyRootFilesystem\":false,\"secrets\":[{\"name\":\"APP_KEY\",\"valueFrom\":\"test\"}],\"volumesFrom\":[]}"

nitrocode commented 3 years ago

@syphernl try the 0.52.0 release tag and it should work.

syphernl commented 3 years ago

@nitrocode Thanks for looking in to this! I'll give it a go :)

syphernl commented 3 years ago

@nitrocode Unfortunately this still isn't working:

Error: Error in function call
  on .terraform/modules/container/main.tf line 21, in locals:
  21:   secrets_as_map      = zipmap(local.secrets_keys, local.secrets_values)
    |----------------
    | local.secrets_keys is (sensitive value)
    | local.secrets_values is (sensitive value)
Call to function "zipmap" failed: panic in function implementation: value is
marked, so must be unmarked first
goroutine 40961 [running]:
runtime/debug.Stack(0xc00c8f7ab8, 0x23d70a0, 0x2c3fd30)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9f
github.com/zclconf/go-cty/cty/function.errorForPanic(...)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/function/error.go:44
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues.func1(0xc00c8f80c8,
0xc00c8f80d8)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/function/function.go:217
+0x7f
panic(0x23d70a0, 0x2c3fd30)
    /usr/local/go/src/runtime/panic.go:969 +0x1b9
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/marks.go:123
github.com/zclconf/go-cty/cty.Value.AsString(0x2ce8fe0, 0xc000056410,
0x25b37c0, 0xc00987d380, 0xc00925ba00, 0xd)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/value_ops.go:1249 +0x4f
github.com/zclconf/go-cty/cty/function/stdlib.glob..func38(0xc00ae4b580, 0x2,
0x2, 0x3dfd420, 0xc00c8f7f18, 0x0, 0x0)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/function/stdlib/collection.go:1171
+0x306
github.com/zclconf/go-cty/cty/function.Function.ReturnTypeForValues(0xc0000a1d10,
0xc00ae4b580, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/function/function.go:221
+0x454
github.com/zclconf/go-cty/cty/function.Function.Call(0xc0000a1d10,
0xc00ae4b580, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /go/pkg/mod/github.com/zclconf/go-cty@v1.7.1/cty/function/function.go:228
+0xb5
github.com/hashicorp/hcl/v2/hclsyntax.(*FunctionCallExpr).Value(0xc00083e870,
0xc00988f880, 0x0, 0xc00c8f9800, 0x1, 0x1, 0x0, 0x0, 0x0)
    /go/pkg/mod/github.com/hashicorp/hcl/v2@v2.8.2/hclsyntax/expression.go:442
+0x10c5
github.com/hashicorp/terraform/lang.(*Scope).EvalExpr(0xc00a282f50, 0x2ce7b60,
0xc00083e870, 0x2ce9020, 0x3dfd420, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /home/circleci/project/project/lang/eval.go:171 +0x1b7
github.com/hashicorp/terraform/terraform.(*BuiltinEvalContext).EvaluateExpr(0xc007080410,
0x2ce7b60, 0xc00083e870, 0x2ce9020, 0x3dfd420, 0x0, 0x0, 0x0, 0x200000003,
0xc004ce1380, ...)
    /home/circleci/project/project/terraform/eval_context_builtin.go:287 +0xbb
github.com/hashicorp/terraform/terraform.(*NodeLocal).Execute(0xc002249ef0,
0x2d24da0, 0xc007080410, 0xc00c978002, 0x250a100, 0x2691400)
    /home/circleci/project/project/terraform/node_local.go:156 +0x71d
github.com/hashicorp/terraform/terraform.(*ContextGraphWalker).Execute(0xc003fbc410,
0x2d24da0, 0xc007080410, 0x7f4f08be8ac8, 0xc002249ef0, 0x0, 0x0, 0x0)
    /home/circleci/project/project/terraform/graph_walk_context.go:127 +0xbc
github.com/hashicorp/terraform/terraform.(*Graph).walk.func1(0x2691400,
0xc002249ef0, 0x0, 0x0, 0x0)
    /home/circleci/project/project/terraform/graph.go:59 +0x962
github.com/hashicorp/terraform/dag.(*Walker).walkVertex(0xc002610120,
0x2691400, 0xc002249ef0, 0xc00ae4b440)
    /home/circleci/project/project/dag/walk.go:387 +0x375
created by github.com/hashicorp/terraform/dag.(*Walker).Update
    /home/circleci/project/project/dag/walk.go:309 +0x1246

nitrocode commented 3 years ago

Did you try the example i gave above and see if that works?

That looks like a terraform bug. Can you create a minimum viable reproducible example for this error?

syphernl commented 3 years ago

Yes I can reproduce it using:

/test_modules/main.tf:

resource "random_string" "password" {
  length  = 32
  special = false
}

output "test_password" {
  value       = random_string.password.result
  description = "Test password"
  sensitive   = true
}

/main.tf:

module "test" {
  source = "./test_module"
}

module "demo_container" {
  source          = "git::https://github.com/cloudposse/terraform-aws-ecs-container-definition.git?ref=0.52.0"
  container_name  = "demo-container"
  container_image = "nginxdemos/hello:latest"

  secrets = [
    {
      name      = "PASSWORD",
      valueFrom = module.test.test_password
    }
  ]

  environment = [
    {
      name  = "APP_ENV",
      value = "production",
    }
  ]

  port_mappings = [
    {
      containerPort = 80
      hostPort      = 80
      protocol      = "tcp"
    }
  ]

  healthcheck = {
    command     = ["CMD-SHELL", "curl -f http://localhost:80/health || exit 1"]
    retries     = 5
    timeout     = 5
    interval    = 30
    startPeriod = 30
  }
}

output "demo_container" {
  value = module.demo_container.json_map_encoded
}

cloudposse / terraform-aws-ecs-container-definition