perpetual change elasticsearch_index_template resource

[sentinelleader]

Hello we are using v2.0.2 release and even in the previous releases we are seeing perpetual change for elasticsearch_index_template resource on everny terraform plan. Plan shows we are trying to set it to null (see the below plan output)

# resource in terraform 

# terraform plan output
...
Terraform detected the following changes made outside of Terraform since the
last "terraform apply":

  # module.logging-infra[0].elasticsearch_index_template.set_log_shard_count[0] has changed
  ~ resource "elasticsearch_index_template" "set_log_shard_count" {
      ~ body = jsonencode(
          ~ {
              - template       = {
                  - aliases  = {
                      - kinfra_logs = {}
                    }
                  - mappings = {
                      - properties = {
                          - timestamp = {
                              - format = "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
                              - type   = "date"
                            }
                          - value     = {
                              - type = "double"
                            }
                        }
                    }
                  - settings = {
                      - index              = {
                          - mapping = {
                              - ignore_malformed = true
                            }
                        }
                      - number_of_replicas = 1
                      - number_of_shards   = 3
                    }
                } -> null
                # (1 unchanged element hidden)
            }
        )
        id   = "log_shard_count"
        name = "log_shard_count"
    }
...
...
Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.logging-infra[0].elasticsearch_index_template.set_log_shard_count[0] will be updated in-place
  ~ resource "elasticsearch_index_template" "set_log_shard_count" {
      ~ body = jsonencode(
          ~ {
              + template       = {
                  + aliases  = {
                      + kinfra_logs = {}
                    }
                  + mappings = {
                      + properties = {
                          + timestamp = {
                              + format = "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
                              + type   = "date"
                            }
                          + value     = {
                              + type = "double"
                            }
                        }
                    }
                  + settings = {
                      + index              = {
                          + mapping = {
                              + ignore_malformed = true
                            }
                        }
                      + number_of_replicas = 1
                      + number_of_shards   = 3
                    }
                }
                # (1 unchanged element hidden)
            }
        )
        id   = "log_shard_count"
        name = "log_shard_count"
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Is this related to https://github.com/phillbaker/terraform-provider-elasticsearch/issues/93 ?

[phillbaker]

Hello, can you please include the following:

• terraform version
• elasticsearch version (and opendistro version if relevant)
• redacted version of the terraform resource configuration

[moritzzimmer]

Maybe related, we have the same problem when using elasticsearch_opendistro_ism_policy and elasticsearch_opendistro_monitor

• Terraform v1.1.6
• AWS OpenSearch Service 1.1
• provider version 2.0.2

In the case of elasticsearch_opendistro_ism_policy, the ism_template part is always displayed as a change set

config example

resource "elasticsearch_opendistro_ism_policy" "ism_policy" {
  for_each = var.ism_policies

  body      = each.value
  policy_id = each.key
}

{
  "policy": {
    "description": "Policy for log indices.",
    "default_state": "hot",
    "ism_template": {
      "index_patterns": ["*"],
      "priority": 0
    },
    "states": []
  }
}

output

# module.log_cluster.elasticsearch_opendistro_ism_policy.ism_policy["default"] will be updated in-place
  ~ resource "elasticsearch_opendistro_ism_policy" "ism_policy" {
      ~ body         = jsonencode(
          ~ {
              ~ policy = {
                  ~ ism_template  = [
                      - {
                          - index_patterns = [
                              - "*",
                            ]
                          - priority       = 0
                        },
                    ] -> {
                      + index_patterns = [
                          + "*",
                        ]
                      + priority       = 0
                    }
                    # (3 unchanged elements hidden)
                }
            }
        )
        id           = "default"
        # (3 unchanged attributes hidden)
    }

[kbennett-v2x]

Also seeing this on elasticsearch_composable_index_template. It finds changes in the index pattern and index settings when there are none.

Terraform v1.2.7 AWS OpenSearch Service 1.3 provider version 2.0.4

~ body = jsonencode(
          ~ {
              ~ index_patterns = [
                  - "j2735_*",
                ] -> "j2735_*"
              ~ template       = {
                  ~ settings = {
                      ~ index = {
                          ~ number_of_replicas = "2" -> 2
                          ~ number_of_shards   = "1" -> 1
                        }
                    }
                    # (1 unchanged element hidden)
                }
                # (3 unchanged elements hidden)
            }
        )
        id   = ""
        name = ""
    }

[etolbakov]

Hello @phillbaker! Fist of all, thank you very much for the great work you are doing on this provider!

Our team also seeing a similar issue with the elasticsearch_index_template resource.

  ~ resource "elasticsearch_index_template" "index_template" {
      ~ body = jsonencode(
          ~ {
              + aliases        = {}
              + mappings       = {}
              + order          = 0
                # (2 unchanged elements hidden)
            }
        )
        id   = "template-a"
        name = "template-a"
    }

We are using Terraform v1.3.7 AWS OpenSearch Service 1.3 provider version 2.0.7

Happy to provide more information if needed. Also will be glad to assist in resolving this issue if it's a known one.

[k-brooks]

I've seen a couple different issues here, might help someone else facing similar issues (as I was)

1. Type conversions number_of_replicas = "2" -> 2 - your configuration is using a string, while the refreshed state has a number. Simply update your body to use the matching types.
2. This was my, and likely the OP's problem - From what I can tell, this resource is using the deprecated _template endpoint, which follows a different format than the _index_template endpoint. Remove the template wrapper in your body and I suspect things will start working. This may be a byproduct of using Opensearch + esclient version detection.