search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.86k stars 9.21k forks source link

Inconsistent final plan when using custom_origin in Cloudfront #11235

Open gnouts opened 4 years ago

gnouts commented 4 years ago

Community Note

Terraform Version

> terraform -v
Terraform v0.12.17
+ provider.aws v2.41.0

Affected Resource(s)

Terraform Configuration Files

locals {
  profile = "default"
  region = "eu-west-3"
  cidr = "10.0.0.0/16"
}

provider "aws" {
  profile    = local.profile
  region     = local.region
}

# this create a VPC with 2 subnets : one public+IGW, one private. they communicate through NAT.
module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  name = "TEST"
  cidr = local.cidr
  azs = ["${local.region}a"]
  public_subnets  = [cidrsubnet(local.cidr,8,1)]
  private_subnets = [cidrsubnet(local.cidr,8,101)]
  enable_nat_gateway = true
  single_nat_gateway = true
  enable_vpn_gateway = false
}

data "aws_iam_instance_profile" "default" {
  name = "aws-elasticbeanstalk-ec2-role"
}

resource "aws_elastic_beanstalk_application" "backend" {
  name        = "TEST"
  description = ""
}

resource "aws_elastic_beanstalk_environment" "backend" {
  name                = "TEST"
  application         = aws_elastic_beanstalk_application.backend.name
  solution_stack_name = "64bit Amazon Linux 2018.03 v2.12.17 running Docker 18.06.1-ce"

  setting {
    resource  = ""
    namespace = "aws:ec2:vpc"
    name      = "VPCId"
    value     = module.vpc.vpc_id
  }
  setting {
    resource  = ""
    namespace = "aws:ec2:vpc"
    name      = "subnets"
    value     = join(",", module.vpc.public_subnets)
  }
  setting {
    resource  = ""
    namespace = "aws:ec2:vpc"
    name      = "elbsubnets"
    value     = join(",", module.vpc.private_subnets)
  }
  setting {
    resource  = ""
    namespace = "aws:autoscaling:launchconfiguration"
    name = "IamInstanceProfile"
    value = data.aws_iam_instance_profile.default.name
  }
}

resource "aws_cloudfront_origin_access_identity" "s3_origin_access_identity" {
  comment = "TEST"
}

resource "aws_s3_bucket" "front" {
  bucket = "terraform-test-front"
  force_destroy = true
  policy = <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {
            "Sid": "CloudFrontRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "${aws_cloudfront_origin_access_identity.s3_origin_access_identity.iam_arn}"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::terraform-test-front/*"
        }
  ]
}
EOF
  website {
    index_document = "index.html"
    error_document = "404.html"
  }
  versioning {
      enabled = false
  }
}

data "aws_elb" "beanstalk_elb" {
  name = aws_elastic_beanstalk_environment.backend.load_balancers[0]
}

resource "aws_cloudfront_distribution" "cdn" {
  enabled      = true
  price_class  = "PriceClass_100"
  http_version = "http2"
  default_root_object = "index.html"
  default_cache_behavior {
    allowed_methods        = ["GET", "HEAD", "DELETE", "OPTIONS", "PATCH", "POST", "PUT"]
    cached_methods         = ["GET", "HEAD"]
    #min_ttl                = "0"
    #default_ttl            = "360"
    #max_ttl                = "1200"
    target_origin_id       = aws_s3_bucket.front.id
    viewer_protocol_policy = "redirect-to-https"
    compress               = true
    forwarded_values {
      query_string = false
      cookies {
        forward = "none"
      }
    }
  }
  # front
  origin {
    origin_id   = aws_s3_bucket.front.id
    domain_name = aws_s3_bucket.front.bucket_regional_domain_name
    s3_origin_config {
      origin_access_identity  = aws_cloudfront_origin_access_identity.s3_origin_access_identity.cloudfront_access_identity_path
    }
  }
  # back
  origin { 
    origin_id   = data.aws_elb.beanstalk_elb.name
    domain_name = data.aws_elb.beanstalk_elb.dns_name
    custom_origin_config {
      http_port              = 80
      https_port             = 443
      origin_protocol_policy = "https-only"
      origin_ssl_protocols   = ["TLSv1", "SSLv3", "TLSv1", "TLSv1.1", "TLSv1.2"]
    }
  }
  custom_error_response {
    error_code            = "404"
    error_caching_min_ttl = "360"
  }
  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }
  viewer_certificate {
    cloudfront_default_certificate = true
  }
}

Debug Output

https://gist.github.com/gnouts/1a7581816e3c79a2d6779c28c733d268

Panic Output

aws_elastic_beanstalk_environment.backend: Modifications complete after 46s [id=e-kpjkj9ngk8]
data.aws_elb.beanstalk_elb: Refreshing state...

Error: Provider produced inconsistent final plan

When expanding the plan for aws_cloudfront_distribution.cdn to include new
values learned so far during apply, provider "aws" produced an invalid new
value for .origin: planned set element
cty.ObjectVal(map[string]cty.Value{"custom_header":cty.SetValEmpty(cty.Object(map[string]cty.Type{"name":cty.String,
"value":cty.String})),
"custom_origin_config":cty.ListValEmpty(cty.Object(map[string]cty.Type{"http_port":cty.Number,
"https_port":cty.Number, "origin_keepalive_timeout":cty.Number,
"origin_protocol_policy":cty.String, "origin_read_timeout":cty.Number,
"origin_ssl_protocols":cty.Set(cty.String)})),
"domain_name":cty.StringVal("terraform-test-front.s3.eu-west-3.amazonaws.com"),
"origin_id":cty.StringVal("terraform-test-front"),
"origin_path":cty.NullVal(cty.String),
"s3_origin_config":cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"origin_access_identity":cty.StringVal("origin-access-identity/cloudfront/E1IP882NL2S666")})})})
does not correlate with any element in actual.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Expected Behavior

CloudFront should have been updated correctly.

Actual Behavior

Terraform crashes and does not complete other action.

Steps to Reproduce

  1. Copy the tf paste above.
  2. terraform init
  3. terraform apply -> this one should be ok.
  4. terraform apply -> Second run crashes !

Important Factoids

In my VPC, I create a public subnet that contains the Beanstalk's ELB and a private subnet that contains the Beanstalk's EC2.

I'm using CloudFront to serve my static front from S3 and redirect API calls to Beanstalk's ELB.

If you comment the second origin (custom origin redirecting to beanstalk) in cloudfront everything work fine.

References

I have reported another issue, which this one is an extension using the same code base, so might be related. https://github.com/terraform-providers/terraform-provider-aws/issues/11217

gnouts commented 4 years ago

I'm now pretty confident this bug is induced by https://github.com/terraform-providers/terraform-provider-aws/issues/11217 If you separate beanstalk and cloudfront in two projects (two folders, two terraform apply, two tfstates) and provide variables in locals, Cloudfront doesn't crash. I tried to used data to pass elb_subnet to cloudfront but somehow the data is marked as changing too. So you have to really use two projects to "lose" the changing state ("known after applied"). Here is the code to test it : https://gist.github.com/gnouts/40a20c986b202633da334a7246e47337

justinretzolk commented 3 years ago

Hey @gnouts 👋 Thank you for taking the time to file this issue and for the additional update. Given that there's been a number of AWS provider releases since your last update, can you confirm whether you're still experiencing this issue?

github-actions[bot] commented 6 days ago

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!