ALB logging fails dependency on bucket policy

[MMarulla]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.18 provider.aws v2.43.0

Affected Resource(s)

• aws_lb
• aws_s3_bucket_policy

Terraform Configuration Files

In main:

resource "aws_s3_bucket_policy" "alb_log_bucket" {
  provider = aws.main
  bucket = aws_s3_bucket.alb_log_bucket.id
  policy =<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::${local.alb_account[var.customer_region]}:root"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::${aws_s3_bucket.alb_log_bucket.id}/*"
    }
  ]
}
EOF
}

In call to module:

module "demo_instance_setup" {
  source = "../modules/standalone_instances"
...
  alb_dependency = [ aws_s3_bucket_policy.alb_log_bucket ]
...

In module:

resource "aws_lb" "alb" {
...
  depends_on = [ var.alb_dependency ]
  access_logs {
    bucket  = var.alb_log_bucket_id
    prefix  = "${var.env}-${each.value}"
    enabled = true
  }
...

Expected Behavior

Without the dependency, the following error is received on apply:

Error: Failure configuring LB attributes: InvalidConfigurationRequest: Access Denied for bucket: << bucket name>>. Please check S3bucket permission status code: 400, request id: 0e870eb5-a774-4985-add3-89370836f7e2

Need to run plan and apply a second time so that the policy is in place before the ALB logging is turned on.

Expected that adding a dependency on the bucket policy would prevent this.

Actual Behavior

With the dependency in place, same error is returned, and the plan/apply have to be run twice to succeed.

Steps to Reproduce

1. Turn on ALB logging
2. Add a bucket policy allowing the AWS account number that is defined for load balancers in that region (see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#access-logging-bucket-permissions)
3. Plan and apply

[arash-bizcover]

Terraform is fundamentally locked-in, just move to Pulumi

[justinretzolk]

Hey @MMarulla 👋 Thank you for taking the time to file this issue. Given that there's been a number of AWS Provider releases since you initially filed it, can you confirm whether you're still experiencing this behavior?

[markrechler]

Have run into a variation of this as well with provider v3.74.0, terraform 1.1.3. Using a depends_on in aws_lb results in a Cycle error though. Was able to work around this by creating access log related buckets and policies via module.

[justinretzolk]

Hey @markrechler 👋 Can you supply a sample of your Terraform configuration? That almost sounds like you've got a depends_on that indicates a resource (resourceA) is dependent on another (resourceB), while that resourceB is somehow dependent on resourceA (perhaps by way of interpolating a value from resourceA?).

[dre2004]

I've also run into this same issue, essentially when you're creating a load balancer with access logs to s3 configured, the load balancer needs a policy to access the bucket.

That policy itself needs the ARN of the load balancer (chicken and egg / cyclic dependency).

data "aws_iam_policy_document" "alb_access_policy" {
  version = "2012-10-17"
  # Load balancer access
  statement {
    principals {
      identifiers = [aws_lb.my_alb.arn]
      type        = "AWS"
    }
    effect  = "Allow"
    actions = [
      "s3:PutObject",
      "s3:PutObjectTagging",
    ]
    resources = [
      aws_s3_bucket.alb_access_logs.arn,
      "${aws_s3_bucket.alb_access_logs.arn}/*"
    ]
  }
}

The only way I feel this could be resolved is if the access_logs configuration was it's own resource. That way the load balancer gets created (giving you the LB arn). The S3 bucket policy is created (giving that LB access to S3) and then allowing the access_logs configuration to be applied.

[a0s]

Found same issue and agree with @dre2004 , how to vote to solution with separated lb_access_logs resource?