Error: ECS Task Definition container_definitions is invalid: Error decoding JSON: json:

[spaceofmiah]

Terraform Version

hashicorp/terraform:0.12.21
"aws" (hashicorp/aws) 2.54.0...
"template" (hashicorp/template) 2.2.0..

Terraform Configuration Files

# container-definitions.json.tpl

[{
        "name": "my_app",
        "image": "${app_image}",
        "essential": true,
        "memoryReservation": 256,
        "environment": [{
                "name": "DEBUG",
                "value": "${debug}"
            },
            {
                "name": "SENTRY_DSN",
                "value": "${sentry}"
            },
            {
                "name": "LOG_LEVEL",
                "value": "${log_level}"
            },
            {
                "name": "SECRET_KEY",
                "value": "${django_secret_key}"
            },
            {
                "name": "SUGGESTION_COUNT",
                "value": "${suggestion_count}"
            },
            {
                "name": "ALLOWED_HOSTS_PROD",
                "value": "${allowed_hosts}"
            },
            {
                "name": "ALLOWED_HOSTS_STAGE",
                "value": "${allowed_hosts}"
            },
            {
                "name": "DEBUG_PROPAGATE_EXCEPTIONS",
                "value": "${debug_propagate}"
            },
            {
                "name": "PROJECT_SUPPORT_ADMIN",
                "value": "${support_admin}"
            },
            {
                "name": "PROJECT_RESOURCE_ADMIN",
                "value": "${support_admin}"
            },
            {
                "name": "DJANGO_SETTINGS_MODULE",
                "value": "${django_settings_module}"
            },

            {
                "name": "DATABASE__ENGINE",
                "values": "${db_engine}"
            },
            {
                "name": "DATABASE__PASSWORD",
                "value": "${db_pass}"
            },
            {
                "name": "DATABASE__HOST",
                "value": "${db_host}"
            },
            {
                "name": "DATABASE__PORT",
                "value": "5432"
            },
            {
                "name": "DATABASE__NAME",
                "value": "${db_name}"
            },
            {
                "name": "DATABASE__USER",
                "value": "${db_user}"
            },

            {
                "name": "EMAIL_PORT",
                "value": "${email_port}"
            },
            {
                "name": "EMAIL_HOST",
                "value": "${email_host}"
            },
            {
                "name": "ADMIN_EMAILS",
                "value": "${admin_emails}"
            },
            {
                "name": "EMAIL_USE_TLS",
                "value": "${email_use_tls}"
            },
            {
                "name": "EMAIL_HOST_USER",
                "value": "${email_host_user}"
            },
            {
                "name": "DEFAULT_FROM_EMAIL",
                "value": "${default_from_email}"
            },
            {
                "name": "EMAIL_HOST_PASSWORD",
                "value": "${email_host_password}"
            },

            {
                "name": "LINODE_BUCKET",
                "value": "${linode_bucket}"
            },
            {
                "name": "LINODE_BUCKET_REGION",
                "value": "${linode_bucket_region}"
            },
            {
                "name": "LINODE_BUCKET_ACCESS_KEY",
                "value": "${linode_bucket_access_key}"
            },
            {
                "name": "LINODE_BUCKET_SECRET_KEY",
                "value": "${linode_bucket_secret_key}"
            },

            {
                "name": "CORS_ALLOW_HEADERS",
                "value": "${cors_allow_headers}"
            },

            {
                "name": "ELK_VERSION",
                "value": "${elk_version}"
            },
            {
                "name": "ELASTIC_USERNAME",
                "value": "${elastic_username}"
            },
            {
                "name": "ELASTIC_PASSWORD",
                "value": "${elastic_password}"
            },
            {
                "name": "ELASTIC_NODE_NAME",
                "value": "${elastic_node_name}"
            },
            {
                "name": "ELASTIC_CLUSTER_NAME",
                "value": "${elastic_cluster_name}"
            },

            {
                "name": "SENDGRID_API_KEY",
                "value": "${sendgrid_api_key}"
            },
            {
                "name": "SENDGRID_LIST_ID",
                "value": "${sendgrid_list_id}"
            }
        ],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "my_app"
            }
        },
        "portMappings": [{
            "containerPort": 9000,
            "hostPort": 9000
        }]
    },
    {
        "essential": true,
        "name": "my_app_elastic",
        "image": "elasticsearch:8.0.0",
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "elastic-search"
            }
        },
        "environment": [{
                "name": "discovery.seed_hosts",
                "value": []
            },
            {
                "name": "bootstrap.memory_lock",
                "value": true
            },
            {
                "name": "discovery.type",
                "value": "single-node"
            },
            {
                "name": "xpack.security.enabled",
                "value": false
            },
            {
                "name": "ELASTIC_USERNAME",
                "value": "${elastic_username}"
            },
            {
                "name": "ELASTIC_PASSWORD",
                "value": "${elastic_password}"
            },
            {
                "name": "ELASTIC_NODE_NAME",
                "value": "${elastic_node_name}"
            },
            {
                "name": "ES_JAVA_OPTS",
                "value": "Xmx${elastic_heap} -Xms${elastic_heap}"
            }
        ],
        "portMappings": [{
            "containerPort": 9200,
            "hostPort": 9200
        }]
    },
    {
        "image": "redis",
        "essential": true,
        "name": "my_app_redis",
        "portMappings": [{
            "containerPort": 6379,
            "hostPort": 6379
        }],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "redis"
            }
        }
    },
    {
        "image": "${app_image}",
        "name": "my_app_celery_beat",
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "celery-beat"
            }
        },
        "command": ["celery", "-A", "my_app", "beat", "-l", "info", "-s", "/code/celerybeat/celerybeat-schedule"]
    },
    {
        "name": "proxy",
        "image": "${proxy_image}",
        "essential": true,
        "portMappings": [{
            "containerPort": 8000,
            "hostPort": 8000
        }],
        "memoryReservation": 256,
        "environment": [{
                "name": "APP_HOST",
                "value": "127.0.0.1"
            },
            {
                "name": "APP_PORT",
                "value": "9000"
            },
            {
                "name": "LISTEN_PORT",
                "value": "8000"
            }
        ],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "proxy"
            }
        }
    }
]

# ecs.tf

data "template_file" "api_container_definitions" {
  template = file("./templates/ecs/container-definitions.json.tpl")

  vars = {
    app_image   = var.ecr_image_api
    proxy_image = var.ecr_image_proxy

    debug                  = var.debug
    sentry                 = var.sentry
    log_level              = var.log_level
    django_secret_key      = var.django_secret_key
    django_settings_module = var.django_settings_module

    suggestion_count = var.suggestion_count
    debug_propagate  = var.debug_propagate
    support_admin    = var.support_admin

    db_engine = "django.db.backends.postgresql"
    db_host   = aws_db_instance.main.address
    db_name   = aws_db_instance.main.name
    db_user   = aws_db_instance.main.username
    db_pass   = aws_db_instance.main.password

    log_group_name   = aws_cloudwatch_log_group.ecs_task_logs.name
    log_group_region = data.aws_region.current.name
    allowed_hosts    = "*"

    email_port          = var.email_port
    email_host          = var.email_host
    admin_emails        = var.admin_emails
    email_use_tls       = var.email_use_tls
    email_host_user     = var.email_host_user
    default_from_email  = var.default_from_email
    email_host_password = var.email_host_password

    linode_bucket            = var.linode_bucket
    linode_bucket_region     = var.linode_bucket_region
    linode_bucket_access_key = var.linode_bucket_access_key
    linode_bucket_secret_key = var.linode_bucket_secret_key

    cors_allow_headers = var.cors_allow_headers

    elk_version          = var.elk_version
    elastic_username     = var.elastic_username
    elastic_heap         = var.elastic_heap
    elastic_password     = var.elastic_password
    elastic_node_name    = var.elastic_node_name
    elastic_cluster_name = var.elastic_cluster_name

    sendgrid_api_key = var.sendgrid_api_key
    sendgrid_list_id = var.sendgrid_list_id
  }
}

resource "aws_ecs_task_definition" "api" {                      # <--- this is line 94 pointed at from the error log
  family                   = "${local.prefix}-api"
  container_definitions    = data.template_file.api_container_definitions.rendered
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = 256
  memory                   = 512
  execution_role_arn       = aws_iam_role.task_execution_role.arn
  task_role_arn            = aws_iam_role.app_iam_role.arn

  tags = local.common_tags
}

Expected Behavior

I expected a generated resource plan, showing all the resource to be created or updated

Actual Behavior

I get an error

Error: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal array into Go struct field KeyValuePair.Environment.Value of type string
  on ecs.tf line 94, in resource "aws_ecs_task_definition" "api":
  94: resource "aws_ecs_task_definition" "api" {

Steps to Reproduce

1. terraform init
2. terraform fmt
3. terraform validate
4. terraform plan

Additional Context

This error only comes up when I run terraform plan within my CI pipeline. Running it on my local machine (same config) pass the plan stage.

PS: this is my first time using terraform and I'm following a course which uses the above terraform version. I attempted changing the version and I got a whole bunch of errors.

[trevorrea]

Hi @spaceofmiah

This isn't a bug in the Terraform AWS provider. It's just something wrong in the particular example you're using. It's much simpler to use jsonencode directly in the aws_ecs_task_definition resource rather than abstracting it through using a template file and then using the rendered template file in the aws_ecs_task_definition resource. The end result is the same but doing it that way is much harder to troubleshoot.

The Terraform docs have excellent examples at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition of how to use jsonencode.

resource "aws_ecs_task_definition" "service" {
  family = "service"
  container_definitions = jsonencode([
    {
      name      = "first"
      image     = "service-first"
      cpu       = 10
      memory    = 512
      essential = true
      portMappings = [
        {
          containerPort = 80
          hostPort      = 80
        }
      ]
    },
    {
      name      = "second"
      image     = "service-second"
      cpu       = 10
      memory    = 256
      essential = true
      portMappings = [
        {
          containerPort = 443
          hostPort      = 443
        }
      ]
    }
  ])
}

[justinretzolk]

Since we haven't heard back, and with the above in mind, I'm going to close this issue.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.