aws_flow_log does not save iam_role_arn and explicitly sets default destination options

[TeraDonawa]

Terraform Version

0.13.5

AWS Terraform Provider version

3.74.3

Affected Resource(s)

• aws_flow_log

Terraform Configuration Files

resource "aws_flow_log" "vpc_flowlog" {
  log_destination      = aws_s3_bucket.vpc_flow_log.arn
  log_destination_type = "s3"
  traffic_type         = "REJECT"
  vpc_id               = var.vpc_id
  iam_role_arn         = aws_iam_role.flow_logs_s3_access.arn 
  tags = merge(
    var.default_tags,
    {
      Name = "${var.vpc_name} - flow log"
    }
  )
}

resource "aws_iam_role" "flow_logs_s3_access" {
  name               = "${var.vpc_name}_${var.s3_state_root}_vpc_flow_logs"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
  tags = merge(
    var.default_tags,
    {
      Name = "${var.vpc_name}_${var.s3_state_root}_vpc_flow_Logs"
    }
  )
}
locals {
  flowbucket_arn = aws_s3_bucket.vpc_flow_log.arn
}

resource "aws_iam_role_policy" "flow_logs_s3_access" {
  name   = "${var.vpc_name}_${var.s3_state_root}_vpc_flow_Logs"
  role   = aws_iam_role.flow_logs_s3_access.id
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams"
      ],
      "Effect": "Allow",
      "Resource": "${local.flowbucket_arn}/*"
    }
  ]
}
EOF
}

Expected Behaviour

After performing a terraform apply to an aws_flow_log resource, any subsequent run of terraform plan should show no differences.

Actual Behaviour

We have noticed that the aws_flow_log is always showing a diff, even immediately after an apply. For example, we'll see

-/+ resource "aws_flow_log" "vpc_flowlog" {
      ~ arn                      = "arn:aws:ec2:us-east-1:<redacted>:vpc-flow-log/fl-00bc52141700071f2" -> (known after apply)
      + iam_role_arn             = "arn:aws:iam::<redacted>:role/ctrl18_stg_vpc_flow_logs" # forces replacement
      ~ id                       = "fl-00bc52141700071f2" -> (known after apply)
        log_destination          = "arn:aws:s3:::cct-stg-ctrl18-vpcflowlog-<redacted>-us-east-1"
        log_destination_type     = "s3"
      ~ log_format               = "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}" -> (known after apply)
      + log_group_name           = (known after apply)
        max_aggregation_interval = 600
        tags                     = {
            "Name"          = "ctrl18 - flow log"
            "state_root"    = "stg"
            "t_AppID"       = "SVC02462"
            "t_cost_centre" = "10.436.72210.56010.0000.21"
            "t_dcl"         = "2"
            "t_environment" = "STG"
            "terraformed"   = "true"
        }
        tags_all                 = {
            "Name"          = "ctrl18 - flow log"
            "state_root"    = "stg"
            "t_AppID"       = "SVC02462"
            "t_cost_centre" = "10.436.72210.56010.0000.21"
            "t_dcl"         = "2"
            "t_environment" = "STG"
            "terraformed"   = "true"
        }
        traffic_type             = "REJECT"
        vpc_id                   = "vpc-038c6e34eeb67963e"

      - destination_options {
          - file_format                = "plain-text" -> null
          - hive_compatible_partitions = false -> null
          - per_hour_partition         = false -> null
        }
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Our terraform sets the iam_role_arn, but does not set the destination_options nor the log_group_name

Looking at the corresponding state file, we see iam_role_arn and log_group_name are set to ""

"resources": [
    {
      "mode": "managed",
      "type": "aws_flow_log",
      "name": "vpc_flowlog",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "arn": "arn:aws:ec2:us-east-1:<redacted>:vpc-flow-log/fl-0bd5c6389be20aa89",
            "destination_options": [
              {
                "file_format": "plain-text",
                "hive_compatible_partitions": false,
                "per_hour_partition": false
              }
            ],
            "eni_id": null,
            "iam_role_arn": "",
            "id": "fl-0bd5c6389be20aa89",
            "log_destination": "arn:aws:s3:::cct-stg-ctrl18-vpcflowlog-<redacted>-us-east-1",
            "log_destination_type": "s3",
            "log_format": "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}",
            "log_group_name": "",
            "max_aggregation_interval": 600,
            "subnet_id": null,
            "tags": {
              "Name": "ctrl18 - flow log",
              "state_root": "stg",
              "t_AppID": "SVC02462",
              "t_cost_centre": "10.436.72210.56010.0000.21",
              "t_dcl": "2",
              "t_environment": "STG",
              "terraformed": "true"
            },
            "tags_all": {
              "Name": "ctrl18 - flow log",
              "state_root": "stg",
              "t_AppID": "SVC02462",
              "t_cost_centre": "10.436.72210.56010.0000.21",
              "t_dcl": "2",
              "t_environment": "STG",
              "terraformed": "true"
            },
            "traffic_type": "REJECT",
            "vpc_id": "vpc-038c6e34eeb67963e"
          },
          "private": "bnVsbA==",
          "dependencies": [
            "aws_iam_role.flow_logs_s3_access",
            "aws_s3_bucket.vpc_flow_log"
          ]
        }
      ]
    },

We could deal with having to set the destination_options, but the main issue is with iam_role_arn not being set and log_group_name added as it terraform plan will always show a difference even when there is no change in our Terraform code.

Steps to Reproduce

1. terraform apply
2. terraform plan

Second step will show differences

References

[civitaspo]

I face the same issue. Is there any workaround?

[pdindorf]

There is a workaround - instead of creating and using the IAM role for Flow Logs, configure bucket policies to accept data from AWS service delivery.logs.amazonaws.com. Everything is described here: https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-s3.html#flow-logs-s3-permissions

After applying such bucket policies I could remove the IAM role from the aws_flow_log resource in Terraform without sacrificing any functionality.

Hope this helps!

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.