Fixing _list/shards API for closed indices

gargharsh3134 commented 1 week ago

Description

_list/shards API throws index_closed_exception whenever the cluster has closed indices. This change is to resolve and ignore the closed indices from getting explicitly passed to IndicesStatsAction.

Functional Testing

On a local cluster with 2 indices: open-test-index -> Open index with 1P and 1R. new-test-idx -> closed index with 1P and 2 R.

gkharsh@bcd07443f159 OpenSearch % curl localhost:9200/_cat/indices
green open  open-test-index rvZzxo5BSnK3TFlmlPQMkw 1 1 0 0 416b 208b
green close new-test-idx    eHm4npFbTQ-dfU8abSLJoQ 1 2

Existing _cat/shards behaviour:

Unfiltered default query outputs closed shards without stats.

gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_cat/shards"                                           
open-test-index 0 p STARTED 0 208b 127.0.0.1 data1
open-test-index 0 r STARTED 0 208b 127.0.0.1 data2
new-test-idx    0 p STARTED        127.0.0.1 data1
new-test-idx    0 r STARTED        127.0.0.1 data2
new-test-idx    0 r STARTED        127.0.0.1 data3

Explicitly querying for a concrete closed index, throws error.


gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_cat/shards/new-test-idx"   
{"error":{"root_cause":[{"type":"index_closed_exception","reason":"closed","index":"new-test-idx","index_uuid":"eHm4npFbTQ-dfU8abSLJoQ"}],"type":"index_closed_exception","reason":"closed","index":"new-test-idx","index_uuid":"eHm4npFbTQ-dfU8abSLJoQ"},"status":400}%

Wildcards only match open indices and response is empty if only closed indices match the expression

gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_cat/shards/new-test-idx*"
gkharsh@bcd07443f159 OpenSearch %

Fixed _list/shards behaviour:

Unfiltered default query will output closed shards without stats.

gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_list/shards"             
new-test-idx    0 p STARTED        127.0.0.1 data1
new-test-idx    0 r STARTED        127.0.0.1 data2
new-test-idx    0 r STARTED        127.0.0.1 data3
open-test-index 0 p STARTED 0 208b 127.0.0.1 data1
open-test-index 0 r STARTED 0 208b 127.0.0.1 data2
next_token null

Explicitly querying for a concrete closed index, will output closed shards without stats instead of throwing error.


gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_list/shards/new-test-idx" 
new-test-idx 0 p STARTED   127.0.0.1 data1
new-test-idx 0 r STARTED   127.0.0.1 data2
new-test-idx 0 r STARTED   127.0.0.1 data3
next_token null

Wildcards continue to match only open indices and response is empty if only closed indices match the expression


gkharsh@bcd07443f159 OpenSearch % curl "localhost:9200/_list/shards/new-test-idx*"
next_token null

Check List

[x] Functionality includes testing.
[x] API changes companion pull request created, if applicable.
[x] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions[bot] commented 1 week ago

:x: Gradle check result for c62e5f27793b7ae03ce171b3b51c6553cb5004ab: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 1 week ago

:x: Gradle check result for 691803cb2479620a5886ac0e32731ed8bdd5057b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 1 week ago

:white_check_mark: Gradle check result for 4663f2b8cd08f3b79508c7d81a9a77bab890f6ca: SUCCESS

codecov[bot] commented 1 week ago

Codecov Report

Attention: Patch coverage is 50.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 72.13%. Comparing base (ac44300) to head (06a1118).

Files with missing lines	Patch %	Lines
...admin/cluster/shards/TransportCatShardsAction.java	0.00%	2 Missing :warning:
.../org/opensearch/action/support/IndicesOptions.java	75.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #16606 +/- ## ============================================ - Coverage 72.14% 72.13% -0.02% - Complexity 65182 65252 +70 ============================================ Files 5318 5318 Lines 303908 303914 +6 Branches 43974 43975 +1 ============================================ - Hits 219267 219221 -46 - Misses 66643 66744 +101 + Partials 17998 17949 -49 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

github-actions[bot] commented 2 days ago

:white_check_mark: Gradle check result for 06a11183b964d874554c6d657db9f509ed8eb6a3: SUCCESS

opensearch-project / OpenSearch