Add data source for index alias

[rm-hull]

Hi @phillbaker, in the rollover guide, it talks about setting an initial index which has the is_write_index = true property set so that the alias is associated with the first index:

resource "elasticsearch_index" "test" {
  name               = "test-000001"
  number_of_shards   = 1
  number_of_replicas = 1
  aliases = jsonencode({
    "test" = {
      "is_write_index" = true
    }
  })

  depends_on = [elasticsearch_index_template.test]
}

I've done exactly this, but as the rollovers occur, test-000002, 000003 ... (or whatever) is the current write index, and so we see an error when the terraform is subsequently applied:

Error: elastic: Error 500 (Internal Server Error): 
alias [test] has more than one write index [test-000001,test-000015] [type=illegal_state_exception]

I was considering conditionally setting is_write_index depending on whether an index alias was already in place. I think this would need a terraform data source adding to the provider that calls GET _alias/{index-name}, eg.

This returns a JSON payload:

{
  "txsearch.transaction-summary-000026" : {
    "aliases" : {
      "txsearch.transaction-summary" : {
        "is_write_index" : false
      }
    }
  },
  "txsearch.transaction-summary-000021" : {
    "aliases" : {
      "txsearch.transaction-summary" : {
        "is_write_index" : false
      }
    }
  },
  "txsearch.transaction-summary-000029" : {
    "aliases" : {
      "txsearch.transaction-summary" : {
        "is_write_index" : true
      }
    }
  },

--- 8< --- snipped for brevity --- 8< ---

I envisage the data source would be used as follows:

data "elasticsearch_index_alias" "my_index_alias" {
  alias = "txsearch.transaction_summary"
}

resource "elasticsearch_index" "test" {
  name               = "txsearch.transaction_summary-000001"
  number_of_shards   = 1
  number_of_replicas = 1
  aliases = jsonencode({
    "txsearch.transaction_summary" = {
      "is_write_index" = !data.elasticsearch_index_alias.my_index_alias.has_write_index
    }
  })

  depends_on = [elasticsearch_index_template.test]
}

Attributes the data source might support:

• has_write_index: true/false - only true if JSON payload has one entry where is_write_index to be true
• current_write_index: the index name from the above JSON where is_write_index is true
• associated_indices: list of the indexes that have been aliased

As before, happy to supply a PR to cover this functionality .. unless of course there is already a way to circumvent the HTTP 500 error ?

[phillbaker]

Thanks for the notes and investigation. Can you also please include the following:

• terraform version
• provider version
• elasticsearch version (and opendistro version if relevant) or opensearch version

I was considering conditionally setting is_write_index depending on whether an index alias was already in place. I think this would need a terraform data source adding to the provider that calls GET _alias/{index-name}, eg.

Hm, I think the tricky part is that this changes over time, so that the Elasticsearch state doesn't match the terraform state.

I've done exactly this, but as the rollovers occur, test-000002, 000003 ... (or whatever) is the current write index, and so we see an error when the terraform is subsequently applied

I think this 500 is worth investigating (and fixing). This functionality was introduced in https://github.com/phillbaker/terraform-provider-elasticsearch/pull/96, and I think it should account for this behavior. I'm wondering if something changed between versions of ES that made the original solution now throw a 500?

[rm-hull]

Terraform version: 1.1.9 Provider version: 2.0.1 Opensearch version: AWS 1.2 (latest patches applied, which I believe to be 1.2.3)