I've discovered that the current version of curator_cli (version 8.0.10) will throw an Exception on forcemerge when the index in question is already at/below the requested number of segments per shard.
Expected Behavior
The curator_cli program should filter out indices as candidates to forcemerge when they are already at/below the requested number of segments per shard.
Actual Behavior
The curator_cli program throws an exception and exits with a nonzero exit code when the specified index already has the requested number of segments per shard.
Steps to Reproduce the Problem
## Verify that the index in question only has one sesgment per shard
$ curl -sS -XGET http://localhost:9200/cmulogstash-2024.02.21/_segments | jq '[ .. | .shards? // empty | to_entries[] | { "key":"shard \(.key)", "value":.value[0].num_committed_segments }] | flatten | from_entries'
{
"shard 0": 1,
"shard 1": 1
}
## Try to perform a forcemerge on the index
$ curator_cli --loglevel DEBUG --request_timeout 14400 forcemerge --ignore_empty_list --max_num_segments 1 --filter_list '[{"filtertype":"pattern","kind":"prefix","value":"cmulogstash-2024.02.21"}]'
[ ... ]
2024-02-27 10:00:45,867 DEBUG curator.indexlist needs_data:266 Indices: ['cmulogstash-2024.02.21'], Fields: ['age', 'number_of_replicas', 'number_of_shards', 'routing']
2024-02-27 10:00:45,867 DEBUG curator.indexlist indices_exist:201 BEGIN indices_exist
2024-02-27 10:00:45,874 DEBUG curator.indexlist needs_data:281 These indices need data in index_info: ['cmulogstash-2024.02.21']
2024-02-27 10:00:45,875 DEBUG curator.indexlist data_getter:224 BEGIN data_getter
2024-02-27 10:00:45,875 DEBUG curator.indexlist indices_exist:201 BEGIN indices_exist
2024-02-27 10:00:45,881 DEBUG curator.indexlist get_index_settings:310 Getting index settings -- END
2024-02-27 10:00:45,881 DEBUG curator.indexlist filter_closed:802 Filtering closed indices
2024-02-27 10:00:45,881 DEBUG curator.indexlist get_index_state:320 Getting index state -- BEGIN
2024-02-27 10:00:45,881 DEBUG curator.indexlist empty_list_check:400 Checking for empty list
2024-02-27 10:00:45,882 DEBUG curator.indexlist needs_data:266 Indices: ['cmulogstash-2024.02.21'], Fields: ['state']
2024-02-27 10:00:45,882 DEBUG curator.indexlist indices_exist:201 BEGIN indices_exist
2024-02-27 10:00:45,887 DEBUG curator.indexlist needs_data:279 Always check open/close for all passed indices
2024-02-27 10:00:45,887 DEBUG curator.indexlist needs_data:281 These indices need data in index_info: ['cmulogstash-2024.02.21']
2024-02-27 10:00:45,891 DEBUG curator.indexlist empty_list_check:400 Checking for empty list
2024-02-27 10:00:45,892 DEBUG curator.indexlist working_list:411 Generating working list of indices
2024-02-27 10:00:45,892 DEBUG curator.indexlist filter_closed:808 Index cmulogstash-2024.02.21 state: open
2024-02-27 10:00:45,892 DEBUG curator.indexlist __actionable:41 Index cmulogstash-2024.02.21 is actionable and remains in the list.
2024-02-27 10:00:45,892 DEBUG curator.indexlist get_segment_counts:387 Getting index segment counts
2024-02-27 10:00:45,892 DEBUG curator.indexlist empty_list_check:400 Checking for empty list
2024-02-27 10:00:45,892 DEBUG curator.indexlist data_getter:224 BEGIN data_getter
2024-02-27 10:00:45,892 DEBUG curator.indexlist indices_exist:201 BEGIN indices_exist
2024-02-27 10:00:45,898 DEBUG curator.indexlist working_list:411 Generating working list of indices
2024-02-27 10:00:45,898 DEBUG curator.indexlist __not_actionable:44 Index cmulogstash-2024.02.21 is not actionable, removing from list.
2024-02-27 10:00:45,898 DEBUG curator.indexlist __excludify:63 Removed from actionable list: cmulogstash-2024.02.21 has 2 shard(s) + 1 replica(s) with a sum total of 4 segments.
2024-02-27 10:00:45,899 DEBUG curator.indexlist empty_list_check:400 Checking for empty list
2024-02-27 10:00:45,899 CRITICAL curator.cli_singletons.cli_action.forcemerge do_singleton_action:224 Failed to complete action: forcemerge. <class 'Exception'>:
Specifications
Version: 8.0.10
Platform: Ubuntu 22.04 (it shouldn't matter)
Subsystem: curator / forcemerge
Context (Environment)
I have a script that runs nightly to forcemerge cold indices down to a single segment. It depends on script exit codes to check for errors/issues with Elastic index processed. Per the discussion in the Elastic forum (see the link at the beginning of this issue), I'm putting in a feature request to ask that curator_cli not error out on indices that are already at or below the requested number of segments per shard.
This issue has already been discussed on the Elastic forum: https://discuss.elastic.co/t/curator-forcemerge-exception/354245
To submit a bug or report an issue
I've discovered that the current version of curator_cli (version 8.0.10) will throw an Exception on forcemerge when the index in question is already at/below the requested number of segments per shard.
Expected Behavior
The curator_cli program should filter out indices as candidates to forcemerge when they are already at/below the requested number of segments per shard.
Actual Behavior
The curator_cli program throws an exception and exits with a nonzero exit code when the specified index already has the requested number of segments per shard.
Steps to Reproduce the Problem
Specifications
Context (Environment)
I have a script that runs nightly to forcemerge cold indices down to a single segment. It depends on script exit codes to check for errors/issues with Elastic index processed. Per the discussion in the Elastic forum (see the link at the beginning of this issue), I'm putting in a feature request to ask that curator_cli not error out on indices that are already at or below the requested number of segments per shard.