search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.31k stars 9.49k forks source link

ECS Task Definition container_definitions is invalid: Error decoding JSON: json #31566

Closed spaceofmiah closed 2 years ago

spaceofmiah commented 2 years ago

Terraform Version

hashicorp/terraform:0.12.21

Terraform Configuration Files

# container-definitions.json.tpl

[{
        "name": "my_app",
        "image": "${app_image}",
        "essential": true,
        "memoryReservation": 256,
        "environment": [{
                "name": "DEBUG",
                "value": "${debug}"
            },
            {
                "name": "SENTRY_DSN",
                "value": "${sentry}"
            },
            {
                "name": "LOG_LEVEL",
                "value": "${log_level}"
            },
            {
                "name": "SECRET_KEY",
                "value": "${django_secret_key}"
            },
            {
                "name": "SUGGESTION_COUNT",
                "value": "${suggestion_count}"
            },
            {
                "name": "ALLOWED_HOSTS_PROD",
                "value": "${allowed_hosts}"
            },
            {
                "name": "ALLOWED_HOSTS_STAGE",
                "value": "${allowed_hosts}"
            },
            {
                "name": "DEBUG_PROPAGATE_EXCEPTIONS",
                "value": "${debug_propagate}"
            },
            {
                "name": "PROJECT_SUPPORT_ADMIN",
                "value": "${support_admin}"
            },
            {
                "name": "PROJECT_RESOURCE_ADMIN",
                "value": "${support_admin}"
            },
            {
                "name": "DJANGO_SETTINGS_MODULE",
                "value": "${django_settings_module}"
            },

            {
                "name": "DATABASE__ENGINE",
                "values": "${db_engine}"
            },
            {
                "name": "DATABASE__PASSWORD",
                "value": "${db_pass}"
            },
            {
                "name": "DATABASE__HOST",
                "value": "${db_host}"
            },
            {
                "name": "DATABASE__PORT",
                "value": "5432"
            },
            {
                "name": "DATABASE__NAME",
                "value": "${db_name}"
            },
            {
                "name": "DATABASE__USER",
                "value": "${db_user}"
            },

            {
                "name": "EMAIL_PORT",
                "value": "${email_port}"
            },
            {
                "name": "EMAIL_HOST",
                "value": "${email_host}"
            },
            {
                "name": "ADMIN_EMAILS",
                "value": "${admin_emails}"
            },
            {
                "name": "EMAIL_USE_TLS",
                "value": "${email_use_tls}"
            },
            {
                "name": "EMAIL_HOST_USER",
                "value": "${email_host_user}"
            },
            {
                "name": "DEFAULT_FROM_EMAIL",
                "value": "${default_from_email}"
            },
            {
                "name": "EMAIL_HOST_PASSWORD",
                "value": "${email_host_password}"
            },

            {
                "name": "LINODE_BUCKET",
                "value": "${linode_bucket}"
            },
            {
                "name": "LINODE_BUCKET_REGION",
                "value": "${linode_bucket_region}"
            },
            {
                "name": "LINODE_BUCKET_ACCESS_KEY",
                "value": "${linode_bucket_access_key}"
            },
            {
                "name": "LINODE_BUCKET_SECRET_KEY",
                "value": "${linode_bucket_secret_key}"
            },

            {
                "name": "CORS_ALLOW_HEADERS",
                "value": "${cors_allow_headers}"
            },

            {
                "name": "ELK_VERSION",
                "value": "${elk_version}"
            },
            {
                "name": "ELASTIC_USERNAME",
                "value": "${elastic_username}"
            },
            {
                "name": "ELASTIC_PASSWORD",
                "value": "${elastic_password}"
            },
            {
                "name": "ELASTIC_NODE_NAME",
                "value": "${elastic_node_name}"
            },
            {
                "name": "ELASTIC_CLUSTER_NAME",
                "value": "${elastic_cluster_name}"
            },

            {
                "name": "SENDGRID_API_KEY",
                "value": "${sendgrid_api_key}"
            },
            {
                "name": "SENDGRID_LIST_ID",
                "value": "${sendgrid_list_id}"
            }
        ],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "my_app"
            }
        },
        "portMappings": [{
            "containerPort": 9000,
            "hostPort": 9000
        }]
    },
    {
        "essential": true,
        "name": "my_app_elastic",
        "image": "elasticsearch:8.0.0",
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "elastic-search"
            }
        },
        "environment": [{
                "name": "discovery.seed_hosts",
                "value": []
            },
            {
                "name": "bootstrap.memory_lock",
                "value": true
            },
            {
                "name": "discovery.type",
                "value": "single-node"
            },
            {
                "name": "xpack.security.enabled",
                "value": false
            },
            {
                "name": "ELASTIC_USERNAME",
                "value": "${elastic_username}"
            },
            {
                "name": "ELASTIC_PASSWORD",
                "value": "${elastic_password}"
            },
            {
                "name": "ELASTIC_NODE_NAME",
                "value": "${elastic_node_name}"
            },
            {
                "name": "ES_JAVA_OPTS",
                "value": "Xmx${elastic_heap} -Xms${elastic_heap}"
            }
        ],
        "portMappings": [{
            "containerPort": 9200,
            "hostPort": 9200
        }]
    },
    {
        "image": "redis",
        "essential": true,
        "name": "my_app_redis",
        "portMappings": [{
            "containerPort": 6379,
            "hostPort": 6379
        }],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "redis"
            }
        }
    },
    {
        "image": "${app_image}",
        "name": "my_app_celery_beat",
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "celery-beat"
            }
        },
        "command": ["celery", "-A", "my_app", "beat", "-l", "info", "-s", "/code/celerybeat/celerybeat-schedule"]
    },
    {
        "name": "proxy",
        "image": "${proxy_image}",
        "essential": true,
        "portMappings": [{
            "containerPort": 8000,
            "hostPort": 8000
        }],
        "memoryReservation": 256,
        "environment": [{
                "name": "APP_HOST",
                "value": "127.0.0.1"
            },
            {
                "name": "APP_PORT",
                "value": "9000"
            },
            {
                "name": "LISTEN_PORT",
                "value": "8000"
            }
        ],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "${log_group_name}",
                "awslogs-region": "${log_group_region}",
                "awslogs-stream-prefix": "proxy"
            }
        }
    }
]

# ecs.tf

data "template_file" "api_container_definitions" {
  template = file("./templates/ecs/container-definitions.json.tpl")

  vars = {
    app_image   = var.ecr_image_api
    proxy_image = var.ecr_image_proxy

    debug                  = var.debug
    sentry                 = var.sentry
    log_level              = var.log_level
    django_secret_key      = var.django_secret_key
    django_settings_module = var.django_settings_module

    suggestion_count = var.suggestion_count
    debug_propagate  = var.debug_propagate
    support_admin    = var.support_admin

    db_engine = "django.db.backends.postgresql"
    db_host   = aws_db_instance.main.address
    db_name   = aws_db_instance.main.name
    db_user   = aws_db_instance.main.username
    db_pass   = aws_db_instance.main.password

    log_group_name   = aws_cloudwatch_log_group.ecs_task_logs.name
    log_group_region = data.aws_region.current.name
    allowed_hosts    = "*"

    email_port          = var.email_port
    email_host          = var.email_host
    admin_emails        = var.admin_emails
    email_use_tls       = var.email_use_tls
    email_host_user     = var.email_host_user
    default_from_email  = var.default_from_email
    email_host_password = var.email_host_password

    linode_bucket            = var.linode_bucket
    linode_bucket_region     = var.linode_bucket_region
    linode_bucket_access_key = var.linode_bucket_access_key
    linode_bucket_secret_key = var.linode_bucket_secret_key

    cors_allow_headers = var.cors_allow_headers

    elk_version          = var.elk_version
    elastic_username     = var.elastic_username
    elastic_heap         = var.elastic_heap
    elastic_password     = var.elastic_password
    elastic_node_name    = var.elastic_node_name
    elastic_cluster_name = var.elastic_cluster_name

    sendgrid_api_key = var.sendgrid_api_key
    sendgrid_list_id = var.sendgrid_list_id
  }
}

resource "aws_ecs_task_definition" "api" {                      # <--- this is line 94 pointed at from the error log
  family                   = "${local.prefix}-api"
  container_definitions    = data.template_file.api_container_definitions.rendered
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = 256
  memory                   = 512
  execution_role_arn       = aws_iam_role.task_execution_role.arn
  task_role_arn            = aws_iam_role.app_iam_role.arn

  tags = local.common_tags
}

Expected Behavior

I expected a generated resource plan, showing all the resource to be created or updated

Actual Behavior

I get an error 

Error: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal array into Go struct field KeyValuePair.Environment.Value of type string
  on ecs.tf line 94, in resource "aws_ecs_task_definition" "api":
  94: resource "aws_ecs_task_definition" "api" {

Steps to Reproduce

  1. terraform init
  2. terraform fmt
  3. terraform validate
  4. terraform plan

Additional Context

This error only comes up when I run terraform plan within my CI pipeline. Running it on my local machine (same config) pass the plan stage.

PS: this is my first time using terraform and I'm following a course which uses the above terraform version. I attempted changing the version and I got a whole bunch of errors.

jbardin commented 2 years ago

Hello,

This appears to be an issue or question with the AWS provider, not with Terraform itself. You can see existing issues and file a new one in their repository here: https://github.com/hashicorp/terraform-provider-aws/issues. If you have questions about Terraform or the AWS provider, it's better to use the community forum where there are more people ready to help. The GitHub issues here are monitored only by a few core maintainers.

Since this is your first foray into Terraform, I would highly recommend finding more up to date references, since things have changed significantly since v0.12, and it is no longer actively supported. Chances are that unless you are also pinning exact provider versions, the resources may have incompatible changes as well. https://learn.hashicorp.com/terraform might be a good place to start.

Thanks!

github-actions[bot] commented 2 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.