elastic / curator

Curator: Tending your Elasticsearch indices
Other
3.05k stars 635 forks source link

Shrink action incompatible with new data tier roles #1620

Open gnobironts opened 3 years ago

gnobironts commented 3 years ago

Expected Behavior

I am running the following shrink action:

actions:
  1:
    action: shrink
    description: >-
      Cooling daily managed indexes older than 3 months
    options:
      ignore_empty_list: True
      continue_if_exception: True
      #disable_action: True
      shrink_node: frovhelsdata-n01.talentsoft.cloud
      number_of_shards: 1
      shrink_suffix: '-shrink'
      post_allocation:
        allocation_type: include
        key: _tier_preference
        value: data_warm,data_hot
      wait_for_completion: True
      wait_for_rebalance: True
    filters:
   ...

I expect the shrink process to run on the frovhelsdata-n01.talentsoft.cloud data node.

Actual Behavior

When using data tier roles instead of generic data, the shrink action is not able to find a valid data node. I get the following message for all my data nodes (set as data_hot or data_warm). shrink_node: DETERMINISTIC also fails.

2021-10-04 16:11:15,725 INFO      Skipping node "frovhelsdata-n01.talentsoft.cloud": non-data node

frovhelsdata-n01.talentsoft.cloud have the following role set :

node.roles:
- data_hot
- data_content
- ingest

Steps to Reproduce the Problem

Use shrink action on a cluster with no "generic" dedicated data nodes

Specifications

Context (Environment)

On new deployments, usage of new data tier (https://www.elastic.co/guide/en/elasticsearch/reference/7.x/data-tiers.html) seems to be the new standard. Implementing these new data tiers in curator should enable shrink support for non ilm managed indexes.

Detailed Description

Curator should allow data_hot (and maybe also data_warm) nodes for _data_node(self, node_id) in actions.py

untergeek commented 3 years ago

Yeah, this definitely needs to be addressed.