[Bug]: aws_cloudsearch_domain tries to update in-place index_field on upgrade from 3.x

[halostatue]

Terraform Core Version

1.3.3

AWS Provider Version

4.37.0

Affected Resource(s)

• aws_cloudsearch_domain

Expected Behavior

The plan should not indicate aws_cloudsearch_domain changes as none of the resources were changed.

Actual Behavior

$ terraform plan -out plan
 # module.whitelabel-arc-ca-product-search.aws_cloudsearch_domain.product_search will be updated in-place
  ~ resource "aws_cloudsearch_domain" "product_search" {
        id                        = "kinetic-arc-can-dev-product"
        name                      = "kinetic-arc-can-dev-product"
        # (5 unchanged attributes hidden)

      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "product_codes" -> null
          - return    = true -> null
          - search    = true -> null
          - sort      = false -> null
          - type      = "literal-array" -> null
        }
      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "product_id" -> null
          - return    = true -> null
          - search    = false -> null
          - sort      = false -> null
          - type      = "literal" -> null
        }
      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "updated_at" -> null
          - return    = true -> null
          - search    = true -> null
          - sort      = true -> null
          - type      = "date" -> null
        }
      + index_field {
          + analysis_scheme = "_en_default_"
          + facet           = false
          + highlight       = true
          + name            = "category_en"
          + return          = true
          + search          = false
          + sort            = false
          + type            = "text-array"
        }
      - index_field {
          - analysis_scheme = "_en_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "category_en" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = false -> null
          - type            = "text-array" -> null
        }
      + index_field {
          + analysis_scheme = "_en_default_"
          + facet           = false
          + highlight       = true
          + name            = "name_en"
          + return          = true
          + search          = false
          + sort            = true
          + type            = "text"
        }
      - index_field {
          - analysis_scheme = "_en_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "name_en" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = true -> null
          - type            = "text" -> null
        }
      + index_field {
          + analysis_scheme = "_fr_default_"
          + facet           = false
          + highlight       = true
          + name            = "category_fr"
          + return          = true
          + search          = false
          + sort            = false
          + type            = "text-array"
        }
      - index_field {
          - analysis_scheme = "_fr_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "category_fr" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = false -> null
          - type            = "text-array" -> null
        }
      + index_field {
          + analysis_scheme = "_fr_default_"
          + facet           = false
          + highlight       = true
          + name            = "name_fr"
          + return          = true
          + search          = false
          + sort            = true
          + type            = "text"
        }
      - index_field {
          - analysis_scheme = "_fr_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "name_fr" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = true -> null
          - type            = "text" -> null
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "product_codes"
          + return    = true
          + search    = true
          + sort      = false
          + type      = "literal-array"
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "product_id"
          + return    = true
          + search    = false
          + sort      = false
          + type      = "literal"
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "updated_at"
          + return    = true
          + search    = true
          + sort      = true
          + type      = "date"
        }

        # (2 unchanged blocks hidden)
    }

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

cloudsearch-product.zip

This contains the module and an example configuration. It depends on a provider aliased as cloudsearch-domains (cloudsearch exists in only a few domains).

Steps to Reproduce

1. Create the search module under v3.75.3 (the previous versions.tf in the module had >= 3.75, < 5.0; it was locked down as part of an overall upgrade).
2. Upgrade the provider from v3.75.3 to v4.73 (the previous versions.tf in the module had >= 3.75, < 5.0).
3. terraform plan -out plan
4. See there are changes.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

Note that this is a regression of #23570. Even after the release of v4.6.0, @edwandr reported that the issue had not resolved itself. We are currently working around this with a lifecycle.ignore_changes = [index_field] in the module (as previously suggested.

Would you like to implement a fix?

No

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @halostatue 👋 Thank you for taking the time to raise this! If possible, can you supply debug logs (redacted as needed) so that the team can review those as well?

[halostatue]

This is the run:

$ TF_LOG_PATH=debug.log TF_LOG=debug terraform plan -out plan -target=module.whitelabel-arc-ca-product-search
module.whitelabel-arc-ca-product-search.aws_cloudsearch_domain.product_search: Refreshing state... [id=kinetic-arc-can-dev-product]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.whitelabel-arc-ca-product-search.aws_cloudsearch_domain.product_search will be updated in-place
  ~ resource "aws_cloudsearch_domain" "product_search" {
        id                        = "kinetic-arc-can-dev-product"
        name                      = "kinetic-arc-can-dev-product"
        # (5 unchanged attributes hidden)

      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "product_codes" -> null
          - return    = true -> null
          - search    = true -> null
          - sort      = false -> null
          - type      = "literal-array" -> null
        }
      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "product_id" -> null
          - return    = true -> null
          - search    = false -> null
          - sort      = false -> null
          - type      = "literal" -> null
        }
      - index_field {
          - facet     = false -> null
          - highlight = false -> null
          - name      = "updated_at" -> null
          - return    = true -> null
          - search    = true -> null
          - sort      = true -> null
          - type      = "date" -> null
        }
      + index_field {
          + analysis_scheme = "_en_default_"
          + facet           = false
          + highlight       = true
          + name            = "category_en"
          + return          = true
          + search          = false
          + sort            = false
          + type            = "text-array"
        }
      - index_field {
          - analysis_scheme = "_en_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "category_en" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = false -> null
          - type            = "text-array" -> null
        }
      + index_field {
          + analysis_scheme = "_en_default_"
          + facet           = false
          + highlight       = true
          + name            = "name_en"
          + return          = true
          + search          = false
          + sort            = true
          + type            = "text"
        }
      - index_field {
          - analysis_scheme = "_en_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "name_en" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = true -> null
          - type            = "text" -> null
        }
      + index_field {
          + analysis_scheme = "_fr_default_"
          + facet           = false
          + highlight       = true
          + name            = "category_fr"
          + return          = true
          + search          = false
          + sort            = false
          + type            = "text-array"
        }
      - index_field {
          - analysis_scheme = "_fr_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "category_fr" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = false -> null
          - type            = "text-array" -> null
        }
      + index_field {
          + analysis_scheme = "_fr_default_"
          + facet           = false
          + highlight       = true
          + name            = "name_fr"
          + return          = true
          + search          = false
          + sort            = true
          + type            = "text"
        }
      - index_field {
          - analysis_scheme = "_fr_default_" -> null
          - facet           = false -> null
          - highlight       = true -> null
          - name            = "name_fr" -> null
          - return          = true -> null
          - search          = true -> null
          - sort            = true -> null
          - type            = "text" -> null
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "product_codes"
          + return    = true
          + search    = true
          + sort      = false
          + type      = "literal-array"
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "product_id"
          + return    = true
          + search    = false
          + sort      = false
          + type      = "literal"
        }
      + index_field {
          + facet     = false
          + highlight = false
          + name      = "updated_at"
          + return    = true
          + search    = true
          + sort      = true
          + type      = "date"
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
╷
│ Warning: Resource targeting is in effect
│
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of
│ the changes requested by the current configuration.
│
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from
│ errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.
╵

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan"

debug.log

The attached debug log has been filtered of all resources except those under the module (I do not have a minimal reproduction, and potentially setting one up will be time consuming).

[halostatue]

We are periodically seeing this with 1.3.6 and 4.46.0.

[chadmyers]

I'm on 1.3.6 and 4.30.0 and I'm seeing it.

No matter what I do, I can't seem to get terraform to give me a clean plan on an existing cluster even when I arrange my index_field's to match what the plan says it wants to do.

[halostatue]

It’s worth noting that this does not happen on every cloud search definition. I have ~12 defined via a module that gives a uniform definition and they do not change without explicit changes to the module. I have one that consistently reorders and takes an additional 30 minutes. Fortunately, this one is on my smallest infra state, so the extra 30 minutes is not typically harmful in this case alone.

[dariomarquezuala]

I'm on 1.3.2 and 4.56.0 and I'm seeing it.