[ILM] Rollover alias doesn't work with {index} placeholder

[marius-dr]

Kibana version: 6.6.0

Original install method (e.g. download page, yum, from source, etc.): BC1

Describe the bug: One of the options when defining index aliases in an index template uses the {index} placeholder, allowing for creating specific aliases for each index. Ex: for index test, the {index}-alias would get you the test-alias index as an alias. If you specify the alias as {index}-alias when assigning a lifecycle policy to an index template, it will normalize the value and use it as a string, hence it will fail when trying to rollover.

Steps to reproduce:

1. Create an index template with a placeholder alias. Second example on this page: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
2. Create a rollover-enabled lifecycle policy and assign it to this template.
3. Create an index that matches the template and check for the policy error.

Expected behavior: Best case scenario would be translate the placeholder when applying the policy, but another temporary option would be to warn that it doesn't work when setting a placeholder-like alias.

Screenshots (if relevant):

[elasticmachine]

Pinging @elastic/es-ui

[bmcconaghy]

I would argue this is a bug in the way the rollover_alias works for the Elasticsearch implementation of ILM, not a UI bug. We could consider adding a warning as you mention, but the "best case scenario" fix would fall on ES.

[yaronp68]

++ for ES API bug

[jakelandis]

I don't think ILM / rollover is intended to work by defining an alias in the index template. Per the examples [1], only the setting "index.lifecycle.rollover_alias" is setup in the index template, but the actual alias is defined directly on the concrete index at time of creation. This is because the hot index behind the alias needs the is_write_index = true. ILM will manage that flag on the concrete aliases (and is how the incoming traffic is moved from one index to another on a rollover).

[1] https://www.elastic.co/guide/en/elasticsearch/reference/6.x/applying-policy-to-template.html

Please let me know if i am mis-understanding the setup the issue incorrectly. I have included some commands to execute a rollover policy that will complete within 1 minute (as opposed to waiting the default 10 minutes).

PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "10s",
        "actions": {
          "rollover": {
            "max_age": "30s"
          }
        }
      }
    }
  }
}
PUT /_cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "5s",
    "logger.org.elasticsearch.xpack.indexlifecycle": "TRACE"
  }
}
PUT _template/test_template
{
  "index_patterns": [
    "my_index*"
  ],
  "settings": {
    "index.lifecycle.name": "my_policy",
    "index.lifecycle.rollover_alias": "my_index_alias"
  }
}
DELETE my_index-*
PUT my_index-000001
{
  "aliases": {
    "my_index_alias": {
      "is_write_index": true
    }
  }
}
GET */_ilm/explain

[gsagwan]

anyway we can automate this, as we have multiple indexes and they are rolled up every day, however we would also like to rollover depending on the size.

[yuliacech]

This is related to an Elasticsearch issue 36627 that was closed in favor of using a data stream for such a use case. I'm going to change the 'bug' label to an 'enhancement', we could add a warning when the alias is configured using a placeholder.

[JahanviAcquia]

We also have a use case of using indices and not data streams as documents are not time base events. I need a mechanism to use rollover_alias. and allowing {index} with rollover alias will help us with multiple indices. This is required feature and very helpful one. Do we have timeline of this enhancement

[elasticmachine]

Pinging @elastic/kibana-management (Team:Kibana Management)