fdmsantos commented 5 months ago

Terraform Core Version

1.7.2

AWS Provider Version

5.42.0

Affected Resource(s)

aws_opensearchserverless_lifecycle_policy

Expected Behavior

Terraform update data lifecycle policy

Actual Behavior

Terraform gives error when try update data lifecycle policy

Relevant Error/Panic Output Snippet

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to module.opensearch_serverless.aws_opensearchserverless_lifecycle_policy.this[0], provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: .policy_version:
│ was cty.StringVal("MTcxMTM1OTI3OTI2MV8x"), but now cty.StringVal("MTcxMTM2MTU2MDA4MV8y").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

Terraform Configuration Files

resource "aws_opensearchserverless_security_policy" "example" { name = "example" type = "encryption" description = "encryption security policy for example-collection" policy = jsonencode({ Rules = [ { Resource = [ "collection/example-collection" ], ResourceType = "collection" } ], AWSOwnedKey = true }) }

Steps to Reproduce

terraform apply change policy terraform apply

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 5 months ago

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

acwwat commented 3 months ago

@fdmsantos I am unable to reproduce the problem with the latest version of the provider. I've tried changing the policy in different ways, and policy_version updated properly as I expect. The resource code also seems to be correct with handling the computed value. What I suspect is that the migration to autoflex via #37085 might have changed the logic and addressed the issue as a result. Please try the Terraform AWS Provider v5.47.0 or later and see if it fixes your issue.

thda commented 1 month ago

I am able to reproduce with a recent version (5.59)

mstrisoline commented 1 month ago

- Installing hashicorp/aws v5.61.0...
- Installed hashicorp/aws v5.61.0 (signed by HashiCorp)

Re-running seems to work, but all changes introduce an initial failure on apply.

workmanw commented 3 weeks ago

We get this problem intermittently and I'm not completely sure why.

Error

module.common.aws_opensearchserverless_access_policy.data_access_policy: Modifying... [id=dev-core-search]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.common.aws_opensearchserverless_access_policy.data_access_policy,
│ provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an
│ unexpected new value: .policy_version: was
│ cty.StringVal("MTcyMTMxODM4NjY3NV[82](https://..../service-core-search/-/jobs/41819046#L82)"), but now
│ cty.StringVal("MTcyMzc1MjM2MTk2MF[83](https://...../service-core-search/-/jobs/41819046#L83)").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

hashicorp/aws: v5.61.0 terraform: 1.9.3

TF definition

locals {
  collection_name = "dev-core-search"
}

resource "aws_opensearchserverless_collection" "collection" {
  name = local.collection_name
  type = "SEARCH"

  depends_on = [aws_opensearchserverless_security_policy.encryption_policy]
}

resource "aws_opensearchserverless_security_policy" "encryption_policy" {
  name        = local.collection_name
  type        = "encryption"
  description = "encryption policy for ${local.collection_name}"
  policy      = jsonencode({
    Rules = [
      {
        Resource     = ["collection/${local.collection_name}"],
        ResourceType = "collection"
      }
    ],
    AWSOwnedKey = true
  })
}

resource "aws_opensearchserverless_security_policy" "network_policy" {
  name        = local.collection_name
  type        = "network"
  description = "public access for dashboard, VPC access for collection endpoint"
  policy      = jsonencode([
    {
      Description = "VPC access for collection endpoint",
      Rules       = [
        {
          ResourceType = "collection",
          Resource     = [
            "collection/${local.collection_name}"
          ]
        }
      ],
      AllowFromPublic = true
    },
    {
      Description = "Public access for dashboards",
      Rules       = [
        {
          ResourceType = "dashboard"
          Resource     = [
            "collection/${local.collection_name}"
          ]
        }
      ],
      AllowFromPublic = true
    }
  ])
}

resource "aws_opensearchserverless_access_policy" "data_access_policy" {
  name        = local.collection_name
  type        = "data"
  description = "allow index and collection access"
  policy      = jsonencode([
    {
      Rules = [
        {
          ResourceType = "index",
          Resource     = [
            "index/${local.collection_name}/*"
          ],
          Permission = [
            "aoss:CreateIndex",
            "aoss:DeleteIndex",
            "aoss:UpdateIndex",
            "aoss:DescribeIndex",
            "aoss:ReadDocument",
            "aoss:WriteDocument"
          ]
        },
        {
          ResourceType = "collection",
          Resource     = [
            "collection/${local.collection_name}"
          ],
          Permission = [
            "aoss:CreateCollectionItems",
            "aoss:DeleteCollectionItems",
            "aoss:UpdateCollectionItems",
            "aoss:DescribeCollectionItems"
          ]
        }
      ],
      Principal = [
        module.k8s_service_account_role.role.arn
      ]
    },
    {
      Description = "Developer read-only access",
      Rules = [
        {
          ResourceType = "index",
          Resource     = [
            "index/${local.collection_name}/*"
          ],
          Permission = [
            "aoss:DescribeIndex",
            "aoss:ReadDocument"
          ]
        },
        {
          ResourceType = "collection",
          Resource     = [
            "collection/${local.collection_name}"
          ],
          Permission = [
            "aoss:DescribeCollectionItems"
          ]
        }
      ],
      Principal = [
        module.globals.developer_role_arn
      ]
    }
  ])
}

johankarlssonpaf commented 1 week ago

We have the same issue, we can create a policy but not change it.

resource "aws_opensearchserverless_security_policy" "example" {
  name        = "network-example"
  type        = "network"
  description = "Public access"
  policy = jsonencode([
    {
      Description = "Public access to collection and Dashboards endpoint for example collection",
      Rules = [
        {
          ResourceType = "collection",
          Resource = [
            "collection/example-collection"
          ]
        },
        {
          ResourceType = "dashboard"
          Resource = [
            "collection/example-collection"
          ]
        }
      ],
      AllowFromPublic = true
    }
  ])
}

Run terraform init, terraform apply, change the description in the json code of the policy and run terraform apply.

terraform version
Terraform v1.9.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.64.0

terraform apply    
aws_opensearchserverless_security_policy.example: Refreshing state... [id=network-example]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated
with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_opensearchserverless_security_policy.example will be updated in-place
  ~ resource "aws_opensearchserverless_security_policy" "example" {
        id             = "network-example"
        name           = "network-example"
      ~ policy         = jsonencode(
          ~ [
              ~ {
                  ~ Description     = "Public access to collection and Dashboards endpoint for example collection" -> "Public access to collection and Dashboards endpoint for example collection1"
                    # (2 unchanged attributes hidden)
                },
            ]
        )
        # (3 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_opensearchserverless_security_policy.example: Modifying... [id=network-example]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to aws_opensearchserverless_security_policy.example, provider
│ "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: .policy: was
│ cty.StringVal("[{\"AllowFromPublic\":true,\"Description\":\"Public access to collection and Dashboards endpoint
│ for example
│ collection1\",\"Rules\":[{\"Resource\":[\"collection/example-collection\"],\"ResourceType\":\"collection\"},{\"Resource\":[\"collection/example-collection\"],\"ResourceType\":\"dashboard\"}]}]"),
│ but now cty.StringVal("[{\"AllowFromPublic\":true,\"Description\":\"Public access to collection and Dashboards
│ endpoint for example
│ collection\",\"Rules\":[{\"Resource\":[\"collection/example-collection\"],\"ResourceType\":\"collection\"},{\"Resource\":[\"collection/example-collection\"],\"ResourceType\":\"dashboard\"}]}]").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to aws_opensearchserverless_security_policy.example, provider
│ "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: .policy_version: was
│ cty.StringVal("MTcyNDg0MjUwOTQyMV8x"), but now cty.StringVal("MTcyNDg0MjUzMTk3OV8y").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Tested both on ARM and X86 and same error.

joshtrutwin commented 1 day ago

Another report - attempted to update the data access policy with some new values:

Terraform v1.9.2
on linux_amd64
Initializing plugins and modules...
module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy: Modifying... [id=test-dynamo-pipeline-osis]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy,
│ provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an
│ unexpected new value: .policy: was
│ cty.StringVal("[{\"Description\":\"Pipeline Write Access Data
│ Policy\",\"Principal\":[\"arn:aws:iam::<snipped>:role/osis/test-dynamo-pipeline-pipelines-access-role\"],\"Rules\":[{\"Permission\":[\"aoss:CreateIndex\",\"aoss:DescribeIndex\",\"aoss:ReadDocument\",\"aoss:UpdateIndex\",\"aoss:WriteDocument\"],\"Resource\":[\"index/osis-test-collection/*\"],\"ResourceType\":\"index\"}]}]"),
│ but now cty.StringVal("[{\"Description\":\"Pipeline Write Access Data
│ Policy\",\"Principal\":[\"arn:aws:iam::<snipped>:role/osis/test-dynamo-pipeline-pipelines-access-role\"],\"Rules\":[{\"Permission\":[\"aoss:DescribeIndex\",\"aoss:CreateIndex\",\"aoss:UpdateIndex\",\"aoss:WriteDocument\"],\"Resource\":[\"index/osis-test-collection/*\"],\"ResourceType\":\"index\"}]}]").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy,
│ provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an
│ unexpected new value: .policy_version: was
│ cty.StringVal("MTcyNTY1NTc0MjU0NV8x"), but now
│ cty.StringVal("MTcyNTY1ODcwMzA4MV8y").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
Operation failed: failed running terraform apply (exit 1)

joshtrutwin commented 1 day ago

Plan for the post above:

Terraform will perform the following actions:

  # module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy will be updated in-place
  ~ resource "aws_opensearchserverless_access_policy" "aoss_osis_data_access_policy" {
        id             = "test-dynamo-pipeline-osis"
        name           = "test-dynamo-pipeline-osis"
      ~ policy         = jsonencode(
          ~ [
              ~ {
                  ~ Rules       = [
                      ~ {
                          ~ Permission   = [
                              - "aoss:DescribeIndex",
                                "aoss:CreateIndex",
                              + "aoss:DescribeIndex",
                              + "aoss:ReadDocument",
                                "aoss:UpdateIndex",
                                # (1 unchanged element hidden)
                            ]
                            # (2 unchanged attributes hidden)
                        },
                    ]
                    # (2 unchanged attributes hidden)
                },
            ]
        )
        # (3 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

joshtrutwin commented 1 day ago

and when attempting a second apply:

Terraform v1.9.2
on linux_amd64
Initializing plugins and modules...
module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy: Modifying... [id=test-dynamo-pipeline-osis]
╷
│ Error: updating Security Policy (test-dynamo-pipeline-osis)
│ 
│   with module.test_osis_pipeline.aws_opensearchserverless_access_policy.aoss_osis_data_access_policy,
│   on .terraform/modules/test_osis_pipeline/opensearch.tf line 14, in resource "aws_opensearchserverless_access_policy" "aoss_osis_data_access_policy":
│   14: resource "aws_opensearchserverless_access_policy" "aoss_osis_data_access_policy" {
│ 
│ operation error OpenSearchServerless: UpdateAccessPolicy, https response
│ error StatusCode: 400, RequestID: 586d4ae2-5e37-471f-af7a-361f644d744d,
│ ValidationException: No changes detected in policy or policy description
╵
Operation failed: failed running terraform apply (exit 1)

hashicorp / terraform-provider-aws

[Bug]: Error when update Data Lifecycle Policy #36554

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

Community Note