Complete example is not working?

key commented 1 year ago

Description

I am trying to deliver S3 content over CloudFront, and I assumed that S3 authentication would be done using Origin Access Control (OAC), and I worked from the definition in the complete example.

Versions

Module version [Required]: 3.2.1
Terraform version: 1.4.2
Provider version(s): aws 4.63.0

Reproduction Code [Required]

I have commented out some definitions because Route53 is not available in my environment.

# provider "aws" {
#   region = "us-east-1" # CloudFront expects ACM resources in us-east-1 region only
#
#   # Make it faster by skipping something
#   skip_metadata_api_check     = true
#   skip_region_validation      = true
#   skip_credentials_validation = true
#
#   # skip_requesting_account_id should be disabled to generate valid ARN in apigatewayv2_api_execution_arn
#   skip_requesting_account_id = false
# }

locals {
  domain_name = "example.com"
  subdomain   = "cdn"
}

module "cloudfront" {
    source  = "terraform-aws-modules/cloudfront/aws"

  aliases = ["${local.subdomain}.${local.domain_name}"]

  comment             = "My awesome CloudFront"
  enabled             = true
  http_version        = "http2and3"
  is_ipv6_enabled     = true
  price_class         = "PriceClass_All"
  retain_on_delete    = false
  wait_for_deployment = false

  # When you enable additional metrics for a distribution, CloudFront sends up to 8 metrics to CloudWatch in the US East (N. Virginia) Region.
  # This rate is charged only once per month, per metric (up to 8 metrics per distribution).
  create_monitoring_subscription = true

  create_origin_access_identity = true
  origin_access_identities = {
    s3_bucket_one = "My awesome CloudFront can access"
  }

  create_origin_access_control = true
  origin_access_control = {
    s3_oac = {
      description      = "CloudFront access to S3"
      origin_type      = "s3"
      signing_behavior = "always"
      signing_protocol = "sigv4"
    }
  }

#   logging_config = {
#     bucket = module.log_bucket.s3_bucket_bucket_domain_name
#     prefix = "cloudfront"
#   }

  origin = {
    appsync = {
      domain_name = "appsync.${local.domain_name}"
      custom_origin_config = {
        http_port              = 80
        https_port             = 443
        origin_protocol_policy = "match-viewer"
        origin_ssl_protocols   = ["TLSv1", "TLSv1.1", "TLSv1.2"]
      }

      custom_header = [
        {
          name  = "X-Forwarded-Scheme"
          value = "https"
        },
        {
          name  = "X-Frame-Options"
          value = "SAMEORIGIN"
        }
      ]

      origin_shield = {
        enabled              = true
        origin_shield_region = "us-east-1"
      }
    }

    s3_one = { # with origin access identity (legacy)
      domain_name = module.s3_one.s3_bucket_bucket_regional_domain_name
      s3_origin_config = {
        origin_access_identity = "s3_bucket_one" # key in `origin_access_identities`
        # cloudfront_access_identity_path = "origin-access-identity/cloudfront/E5IGQAA1QO48Z" # external OAI resource
      }
    }

    s3_oac = { # with origin access control settings (recommended)
      domain_name           = module.s3_one.s3_bucket_bucket_regional_domain_name
      origin_access_control = "s3_oac" # key in `origin_access_control`
      #      origin_access_control_id = "E345SXM82MIOSU" # external OAС resource
    }
  }

  origin_group = {
    group_one = {
      failover_status_codes      = [403, 404, 500, 502]
      primary_member_origin_id   = "appsync"
      secondary_member_origin_id = "s3_one"
    }
  }

  default_cache_behavior = {
    target_origin_id       = "appsync"
    viewer_protocol_policy = "allow-all"
    allowed_methods        = ["GET", "HEAD", "OPTIONS"]
    cached_methods         = ["GET", "HEAD"]
    compress               = true
    query_string           = true

    # This is id for SecurityHeadersPolicy copied from https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-response-headers-policies.html
    response_headers_policy_id = "67f7725c-6f97-4210-82d7-5512b31e9d03"

    lambda_function_association = {

      # Valid keys: viewer-request, origin-request, viewer-response, origin-response
      viewer-request = {
        lambda_arn   = module.lambda_function.lambda_function_qualified_arn
        include_body = true
      }

      origin-request = {
        lambda_arn = module.lambda_function.lambda_function_qualified_arn
      }
    }
  }

  ordered_cache_behavior = [
    {
      path_pattern           = "/static/*"
      target_origin_id       = "s3_one"
      viewer_protocol_policy = "redirect-to-https"

      allowed_methods = ["GET", "HEAD", "OPTIONS"]
      cached_methods  = ["GET", "HEAD"]
      compress        = true
      query_string    = true

      function_association = {
        # Valid keys: viewer-request, viewer-response
        viewer-request = {
          function_arn = aws_cloudfront_function.example.arn
        }

        viewer-response = {
          function_arn = aws_cloudfront_function.example.arn
        }
      }
    }
  ]

#   viewer_certificate = {
#     acm_certificate_arn = module.acm.acm_certificate_arn
#     ssl_support_method  = "sni-only"
#   }

  custom_error_response = [{
    error_code         = 404
    response_code      = 404
    response_page_path = "/errors/404.html"
    }, {
    error_code         = 403
    response_code      = 403
    response_page_path = "/errors/403.html"
  }]

  geo_restriction = {
    restriction_type = "whitelist"
    locations        = ["NO", "UA", "US", "GB"]
  }

}

######
# ACM
######

# data "aws_route53_zone" "this" {
#   name = local.domain_name
# }

# module "acm" {
#   source  = "terraform-aws-modules/acm/aws"
#   version = "~> 4.0"
#
#   domain_name               = local.domain_name
#   zone_id                   = data.aws_route53_zone.this.id
#   subject_alternative_names = ["${local.subdomain}.${local.domain_name}"]
# }

#############
# S3 buckets
#############

data "aws_canonical_user_id" "current" {}
data "aws_cloudfront_log_delivery_canonical_user_id" "cloudfront" {}

module "s3_one" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "~> 3.0"

  bucket        = "s3-one-${random_pet.this.id}"
  force_destroy = true
}

module "log_bucket" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "~> 3.0"

  bucket = "logs-${random_pet.this.id}"
  acl    = null
#   grant = [{
#     type       = "CanonicalUser"
#     permission = "FULL_CONTROL"
#     id         = data.aws_canonical_user_id.current.id
#     }, {
#     type       = "CanonicalUser"
#     permission = "FULL_CONTROL"
#     id         = data.aws_cloudfront_log_delivery_canonical_user_id.cloudfront.id
#     # Ref. https://github.com/terraform-providers/terraform-provider-aws/issues/12512
#     # Ref. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html
#   }]
  force_destroy = true
}

#############################################
# Using packaged function from Lambda module
#############################################

locals {
  package_url = "https://raw.githubusercontent.com/terraform-aws-modules/terraform-aws-lambda/master/examples/fixtures/python3.8-zip/existing_package.zip"
  downloaded  = "downloaded_package_${md5(local.package_url)}.zip"
}

resource "null_resource" "download_package" {
  triggers = {
    downloaded = local.downloaded
  }

  provisioner "local-exec" {
    command = "curl -L -o ${local.downloaded} ${local.package_url}"
  }
}

module "lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 4.0"

  function_name = "${random_pet.this.id}-lambda"
  description   = "My awesome lambda function"
  handler       = "index.lambda_handler"
  runtime       = "python3.8"

  publish        = true
  lambda_at_edge = true

  create_package         = false
  local_existing_package = local.downloaded

  # @todo: Missing CloudFront as allowed_triggers?

  #    allowed_triggers = {
  #      AllowExecutionFromAPIGateway = {
  #        service = "apigateway"
  #        arn     = module.api_gateway.apigatewayv2_api_execution_arn
  #      }
  #    }
}

##########
# Route53
##########

# module "records" {
#   source  = "terraform-aws-modules/route53/aws//modules/records"
#   version = "~> 2.0"
#
#   zone_id = data.aws_route53_zone.this.zone_id
#
#   records = [
#     {
#       name = local.subdomain
#       type = "A"
#       alias = {
#         name    = module.cloudfront.cloudfront_distribution_domain_name
#         zone_id = module.cloudfront.cloudfront_distribution_hosted_zone_id
#       }
#     },
#   ]
# }

data "aws_iam_policy_document" "s3_policy" {
  # Origin Access Identities
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${module.s3_one.s3_bucket_arn}/static/*"]

    principals {
      type        = "AWS"
      identifiers = module.cloudfront.cloudfront_origin_access_identity_iam_arns
    }
  }

  # Origin Access Controls
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${module.s3_one.s3_bucket_arn}/static/*"]

    principals {
      type        = "Service"
      identifiers = ["cloudfront.amazonaws.com"]
    }

    condition {
      test     = "StringEquals"
      variable = "aws:SourceArn"
      values   = [module.cloudfront.cloudfront_distribution_arn]
    }
  }
}

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = module.s3_one.s3_bucket_id
  policy = data.aws_iam_policy_document.s3_policy.json
}

########
# Extra
########

resource "random_pet" "this" {
  length = 2
}

resource "aws_cloudfront_function" "example" {
  name    = "example-${random_pet.this.id}"
  runtime = "cloudfront-js-1.0"
  code    = file("example-function.js")
}

Yes. Workspace on Terraform Cloud.

Yes.

Expected behavior

The CloudFront Distribution should be created without any errors.

Actual behavior

Terraform apply does not succeed with the following error

Terminal Output Screenshot(s)

│ Error: error creating S3 bucket ACL for logs-innocent-corgi: AccessControlListNotSupported: The bucket does not allow ACLs
│   status code: 400, request id: NJ7835E8980SRGS7, host id: vzPOcfFLVpvQyOaOUV6dZ5HKTVBA8eqNzuEXAZqlDoo0YUwz0x8ZLPn18PllCGiLA8wb9yTr+wQ=
│
│   with module.log_bucket.aws_s3_bucket_acl.this[0],
│   on .terraform/modules/log_bucket/main.tf line 41, in resource "aws_s3_bucket_acl" "this":
│   41: resource "aws_s3_bucket_acl" "this" {
│
╵
╷
│ Error: updating CloudFront Distribution (E41UCPXDJBANV): InvalidLambdaFunctionAssociation: The function must be in region 'us-east-1'. ARN: arn:aws:lambda:ap-northeast-1:183154346239:function:innocent-corgi-lambda:1
│   status code: 400, request id: 2b01c748-ff21-4d8b-a4a2-bd3a016a8b84
│
│   with module.cloudfront.aws_cloudfront_distribution.this[0],
│   on .terraform/modules/cloudfront/main.tf line 27, in resource "aws_cloudfront_distribution" "this":
│   27: resource "aws_cloudfront_distribution" "this" {

Additional context

I believe this is affected by a change in the default settings for AWS S3. This change was made in April 2023.

https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-s3-automatically-enable-block-public-access-disable-access-control-lists-buckets-april-2023/?nc1=h_ls

rberger commented 1 year ago

If you add these to your modules that create s3 buckets:

  control_object_ownership = true
  object_ownership         = "ObjectWriter"

I.E.:

module "s3_one" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "~> 3.11"

  bucket                   = "s3-one-${random_pet.this.id}"
  force_destroy            = true
  control_object_ownership = true
  object_ownership         = "ObjectWriter"
}

Seems to fix it. (Also use a recent version of terraform-aws-modules/s3-bucket/aws)

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

github-actions[bot] commented 1 year ago

This issue was automatically closed because of stale in 10 days

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

antonbabenko commented 9 months ago

This issue has been resolved in version 3.2.2 :tada:

terraform-aws-modules / terraform-aws-cloudfront