tony-aq commented 1 year ago

Terraform Core Version

1.3.2

AWS Provider Version

4.15.1

Affected Resource(s)

aws_cloudfront_distribution resource
aws_cloudfront_cache_policy data source

Expected Behavior

The cloudfront distribution should have been created.

Actual Behavior

Provider error about mismatch of final plan.

Relevant Error/Panic Output Snippet

Error: Provider produced inconsistent final plan

When expanding the plan for
module.media[0].aws_cloudfront_distribution.media-cloudfront to include new
values learned so far during apply, provider
"registry.terraform.io/hashicorp/aws" produced an invalid new value for
.ordered_cache_behavior[0].cache_policy_id: was null, but now
cty.StringVal("658327ea-f89d-4fab-a63d-7e88639e58f6").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Error: Provider produced inconsistent final plan

When expanding the plan for
module.media[0].aws_cloudfront_distribution.media-cloudfront to include new
values learned so far during apply, provider
"registry.terraform.io/hashicorp/aws" produced an invalid new value for
.default_cache_behavior[0].cache_policy_id: was null, but now
cty.StringVal("658327ea-f89d-4fab-a63d-7e88639e58f6").

Terraform Configuration Files

data "aws_cloudfront_cache_policy" "cache-optimized" {
  name = "Managed-CachingOptimized"
}

resource "aws_cloudfront_distribution" "media-cloudfront" {
  origin {
    domain_name = data.aws_s3_bucket.bucket1.bucket_domain_name
    origin_id   = "S3-Bucket"

    s3_origin_config {
      origin_access_identity = var.cloudfront-access-identity-path
    }
  }

  origin {
    domain_name = data.aws_s3_bucket.bucket2.bucket_domain_name
    origin_id   = "S3-Bucket2"

    s3_origin_config {
      origin_access_identity = var.cloudfront-access-identity-path
    }
  }

  enabled = true

  default_cache_behavior {
    allowed_methods = ["GET", "HEAD"]
    cached_methods  = ["GET", "HEAD"]
    target_origin_id = "S3-Bucket"
    cache_policy_id = data.aws_cloudfront_cache_policy.cache-optimized.id
    viewer_protocol_policy = "redirect-to-https"
  }

  ordered_cache_behavior {
    path_pattern     = "/other*"
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "S3-Bucket2"
    cache_policy_id = data.aws_cloudfront_cache_policy.cache-optimized.id
    viewer_protocol_policy = "redirect-to-https"
  }

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  viewer_certificate {
    acm_certificate_arn = aws_acm_certificate_validation.cert-validation.certificate_arn
    ssl_support_method  = "sni-only"
  }

  aliases = [local.cdn-domain-name]
}

Steps to Reproduce

Deploy new cloudfront distribution referencing the aws_cloudfront_cache_policy, provider error.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 1 year ago

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

JimmyBastos commented 1 year ago

Hey @tony-aq! did you manage to solve that?

leonarac commented 1 year ago

i have the same but with the version 4.30

dd84ai commented 1 year ago

versions:

registry.terraform.io/hashicorp/aws = 5.0.1
Terraform v1.4.4
terragrunt version v0.44.5

Applying first time (and many times in a row)

plan is born

  # module.stack.module.sandbox_cf.module.cloudfront.aws_cloudfront_distribution.cdn will be updated in-place
  ~ resource "aws_cloudfront_distribution" "cdn" {
        id                             = "EBOOGAXFN1JW5"
        tags                           = {}
      ~ web_acl_id                     = "arn:aws:wafv2:us-east-1:066731144711:global/webacl/company-firewall/d0f2785b-b999-4d54-87f1-bd7d390015fb" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ default_cache_behavior {
          - cache_policy_id          = "4135ea2d-6df8-44a3-9df3-4b5a84be39ad" -> null
          - origin_request_policy_id = "216adef6-5c7f-47e4-b989-5492eafa07d3" -> null
            # (11 unchanged attributes hidden)
        }

      ~ ordered_cache_behavior {
          - cache_policy_id        = "658327ea-f89d-4fab-a63d-7e88639e58f6" -> null
            # (12 unchanged attributes hidden)
        }

        # (4 unchanged blocks hidden)
    }

plan is next one despite cache_policy_id being defined! Applied multiple times and got same error

    When expanding the plan for
module.stack.module.sandbox_cf.module.cloudfront.aws_cloudfront_distribution.cdn
to include new values learned so far during apply, provider
"registry.terraform.io/hashicorp/aws" produced an invalid new value for
.default_cache_behavior[0].origin_request_policy_id: was null, but now
cty.StringVal("216adef6-5c7f-47e4-b989-5492eafa07d3").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Second time:

I tried to taint them and recreate. They got destroyed, but failed to be created each time with error containing policy cache again

Third time is charming

I got sneaky and applied our older cloudfront terraform code version it still used forwarded_values instead of cache_policy_id and origin_request_policy_id I succesfully created older version of our cloudfront code

Then i tried to apply newest code that uses cache_policies with next plan

  # module.stack.module.sandbox_cf.module.cloudfront.aws_cloudfront_distribution.cdn will be updated in-place
  ~ resource "aws_cloudfront_distribution" "cdn" {
        id                             = "E1SFVNEI93AB6L"
        tags                           = {}
        # (20 unchanged attributes hidden)

      ~ default_cache_behavior {
          + cache_policy_id          = "4135ea2d-6df8-44a3-9df3-4b5a84be39ad"
          + origin_request_policy_id = "216adef6-5c7f-47e4-b989-5492eafa07d3"
            # (11 unchanged attributes hidden)

          - forwarded_values {
              - headers                 = [
                  - "*",
                ] -> null
              - query_string            = true -> null
              - query_string_cache_keys = [] -> null

              - cookies {
                  - forward           = "all" -> null
                  - whitelisted_names = [] -> null
                }
            }
        }

      ~ ordered_cache_behavior {
          + cache_policy_id        = "658327ea-f89d-4fab-a63d-7e88639e58f6"
            # (12 unchanged attributes hidden)

          - forwarded_values {
              - headers                 = [
                  - "*",
                ] -> null
              - query_string            = true -> null
              - query_string_cache_keys = [] -> null

              - cookies {
                  - forward           = "all" -> null
                  - whitelisted_names = [] -> null
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

It was applied succesfully, it updated itself correctly

Fourth time applying:

Out of experiment i destroyed them back and tried to create again I expected to receive same error on creation like in second time, but no it was created succesfully from zero now.

TLDR: Temporal solution

if you recreate with forwarded values instead of cache_policy_id and origin_request_policy_id And then update to version with cache_policy_id and origin_request_policy_id, then problem will be gone and is not longer recreated Observe code in my third time for hints

dd84ai commented 1 year ago

Continuation of https://github.com/hashicorp/terraform-provider-aws/issues/29234#issuecomment-1567447028

Deployed to production without aws_cloudfront_distribution recreation. Just update in place to solution with forwarded_values and revert back via update to using cache_policy_id and origin_request_policy_id once again helped to cure the problem. recreation of resource is not required.

dd84ai commented 1 year ago

Update: same bug returned back to staging and production after this solution at some point of time :smiling_face_with_tear:

npellegrin commented 8 months ago

I am experiencing the issue as well. As a workaround, creating custom cache policies identical to managed policies and attaching it to CloudFront works:

# Same as: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-cache-policies.html#managed-cache-caching-optimized
resource "aws_cloudfront_cache_policy" "caching_optimized" {
  name    = "UserDefined-CachingOptimized"
  comment = "Caching optimize policy"

  default_ttl = 86400    # 24 hours
  max_ttl     = 31536000 # 365 days
  min_ttl     = 1

  parameters_in_cache_key_and_forwarded_to_origin {
    cookies_config {
      cookie_behavior = "none"
    }
    headers_config {
      header_behavior = "none"
    }
    query_strings_config {
      query_string_behavior = "none"
    }

    enable_accept_encoding_gzip   = true
    enable_accept_encoding_brotli = true
  }
}

ZsoltPath commented 8 months ago

This doesn't work for me. Unfortunately we have dozens of distributions. The limit doesn't allow to create a policy for each of them. And If I create a shared one, I get the same error above as with the managed one.

I am experiencing the issue as well. As a workaround, creating custom cache policies identical to managed policies and attaching it to CloudFront works:

# Same as: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-cache-policies.html#managed-cache-caching-optimized
resource "aws_cloudfront_cache_policy" "caching_optimized" {
  name    = "UserDefined-CachingOptimized"
  comment = "Caching optimize policy"

  default_ttl = 86400    # 24 hours
  max_ttl     = 31536000 # 365 days
  min_ttl     = 1

  parameters_in_cache_key_and_forwarded_to_origin {
    cookies_config {
      cookie_behavior = "none"
    }
    headers_config {
      header_behavior = "none"
    }
    query_strings_config {
      query_string_behavior = "none"
    }

    enable_accept_encoding_gzip   = true
    enable_accept_encoding_brotli = true
  }
}

lorelei-rupp-imprivata commented 5 months ago

We hit this as well, because the cache_policy_id is set via a data, its null at plan time if the data has to reload, but then not null at apply time causing the inconsistency

julian-alarcon commented 4 months ago

This bug looks pretty similar to this one: https://github.com/hashicorp/terraform-provider-aws/issues/22467

fahadahammed commented 3 months ago

Faced same. It fails first time. Then if again apply, it successfully completes the tasks.

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.cloudfront.aws_cloudfront_distribution.cdn to include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for .ordered_cache_behavior[0].cache_policy_id: was null, but now
│ cty.StringVal("<ID>").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.cloudfront.aws_cloudfront_distribution.cdn to include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for .ordered_cache_behavior[1].cache_policy_id: was null, but now
│ cty.StringVal("<ID>").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.cloudfront.aws_cloudfront_distribution.cdn to include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for .default_cache_behavior[0].cache_policy_id: was null, but now
│ cty.StringVal("<ID>").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Any way around?

nishit031 commented 2 months ago

Hi @tony-aq

I was facing same issue while I am using terraform cloudfront public ModuleLink

Error: Provider produced inconsistent final plan

When expanding the plan for
module.cloudfront_frontend[0].module.cloudfront.aws_cloudfront_distribution.this[0]
to include new values learned so far during apply, provider
"registry.opentofu.org/hashicorp/aws" produced an invalid new value for
.ordered_cache_behavior[1].cache_policy_id: was null, but now
cty.StringVal("4135ea2d-6df8-44a3-9df3-4b5a84be39ad").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Error: Provider produced inconsistent final plan

When expanding the plan for
module.cloudfront_frontend[0].module.cloudfront.aws_cloudfront_distribution.this[0]
to include new values learned so far during apply, provider
"registry.opentofu.org/hashicorp/aws" produced an invalid new value for
.ordered_cache_behavior[1].origin_request_policy_id: was null, but now
cty.StringVal("b0a8-53d0-40ab-baf2-68738e2966ac").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.
Error: OpenTofu exited with code 1.
Error: Process completed with exit code 1.

I was encountering an issue with my environment, despite it working in another setup. After trying multiple solutions without success, I decided to review the CloudFront public module. During my review, I discovered that the cache_policy_id can take three different values: one from a variable, one from a Terraform data lookup, and a null value.

I realized that I was passing a name instead of the ID and relying on the module to find the ID. Instead of using the module for this purpose, I decided to look up the ID directly within my own Terraform script. By replacing cache_policy_name with cache_policy_id in my script and using the data lookup functionality provided by Terraform, I was able to resolve the issue successfully.

This is I am using at intial stage where I am getting error.

ordered_cache_behavior = [
    {
      path_pattern               = "/api/*"
      target_origin_id           = "${module.api_gateway_agent_desktop_backend.apigatewayv2_api_id}.execute-api.${var.region}.amazonaws.com"
      viewer_protocol_policy     = "allow-all"
      use_forwarded_values       = false
      cache_policy_name            = "Managed-CachingDisabled"
      origin_request_policy_name   = "Managed-AllViewerExceptHostHeader"
      response_headers_policy_name = "Managed-SimpleCORS"
      allowed_methods = [
        "DELETE",
        "GET",
        "HEAD",
        "OPTIONS",
        "PATCH",
        "POST",
        "PUT",
      ]
    }
  ]

And here now I am using with my solution and it is working for me!

ordered_cache_behavior = [
    {
      path_pattern               = "/api/*"
      target_origin_id           = "${module.api_gateway_agent_desktop_backend.apigatewayv2_api_id}.execute-api.${var.region}.amazonaws.com"
      viewer_protocol_policy     = "allow-all"
      use_forwarded_values       = false
      cache_policy_id            = data.aws_cloudfront_cache_policy.cachingdisable.id
      origin_request_policy_id   = data.aws_cloudfront_origin_request_policy.allviewerexcepthostheader.id
      response_headers_policy_id = data.aws_cloudfront_response_headers_policy.simplecors.id
      allowed_methods = [
        "DELETE",
        "GET",
        "HEAD",
        "OPTIONS",
        "PATCH",
        "POST",
        "PUT",
      ]
    }
  ]

 data "aws_cloudfront_cache_policy" "cachingdisable" {
  name = "Managed-CachingDisabled"
}

data "aws_cloudfront_origin_request_policy" "allviewerexcepthostheader" {
  name = "Managed-AllViewerExceptHostHeader"
}

data "aws_cloudfront_response_headers_policy" "simplecors" {
  name = "Managed-SimpleCORS"
}

That's it I hope this solution will work for you guys. You can try this and leave comment if you find it is helpful. Thank you

bt-macole commented 1 month ago

For me, we were reading the policy data resources and creating the distribution as a submodule. When nesting the distribution inside a submodule, the data resources are only read during the apply phase. Moving the data resources to the root module and passing the IDs as variables to the submodule avoids inconsistency issues.

If you have a similar issue you will see something like the below on your plan:

# (depends on a resource or a module with changes pending)
 <= data "aws_cloudfront_cache_policy" "this" {
      + id            = (known after apply)

hashicorp / terraform-provider-aws

[Bug]: Creating new Cloudfront distribution using cache policy ID fails with inconsistent final plan #29234

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

Community Note

versions:

Applying first time (and many times in a row)

Second time:

Third time is charming

Fourth time applying:

TLDR: Temporal solution

Continuation of https://github.com/hashicorp/terraform-provider-aws/issues/29234#issuecomment-1567447028