opensearch-project / index-management

🗃 Automate periodic data operations, such as deleting indices at a certain age or performing a rollover at a certain size
https://opensearch.org/docs/latest/im-plugin/index/
Apache License 2.0
53 stars 108 forks source link

[BUG] ISM should honor `plugins` namespace more strictly #1050

Open r1walz opened 7 months ago

r1walz commented 7 months ago

What is the bug?

ISM setting exists between two namespaces, plugins. and opendistro.. opendistro namespace is reminiscence but still not fully deprecated as some of the settings are required for migration purposes.

ISM utilizes index_state_management.auto_manage setting to get to know which indices to manage. But, since the setting exists under two namespaces plugins. and opendistro., if someone does not want ISM to manage the index, they need to disable setting under both namespaces ...

https://github.com/opensearch-project/index-management/blob/417d0d9c3ac630b720081f3ea383dea26f4a6456/src/main/kotlin/org/opensearch/indexmanagement/IndexManagementPlugin.kt#L499

https://github.com/opensearch-project/index-management/blob/417d0d9c3ac630b720081f3ea383dea26f4a6456/src/main/kotlin/org/opensearch/indexmanagement/IndexManagementPlugin.kt#L537

... as disabling only one is going to violate the following check:

https://github.com/opensearch-project/index-management/blob/417d0d9c3ac630b720081f3ea383dea26f4a6456/src/main/kotlin/org/opensearch/indexmanagement/indexstatemanagement/ManagedIndexCoordinator.kt#L334-L338

How can one reproduce the bug?

Steps to reproduce the behavior: ```sh # create ISM policy $ http PUT :9200/_plugins/_ism/policies/sample policy:='{ "description": "sample policy", "default_state": "hot", "states": [{ "name": "hot", "actions": [{}] }], "ism_template": { "index_patterns": ["sample-idx-*"], "priority": 100 } }' # create index matching policy index pattern $ http PUT :9200/sample-idx-001 [2023-12-01T09:54:54,644][INFO ][o.o.i.i.ManagedIndexCoordinator] [integTest-0] Index [sample-idx-001] matched ISM policy template and will be managed by sample # ISM manages index sample-idx-001 # create index matching policy index pattern with plugins.*.auto_manage disabled $ http PUT :9200/sample-idx-002 settings:='{ "index": { "plugins.index_state_management.auto_manage": "false" } }' [2023-12-01T09:54:54,644][INFO ][o.o.i.i.ManagedIndexCoordinator] [integTest-0] Index [sample-idx-002] matched ISM policy template and will be managed by sample # ISM manages index sample-idx-002 # create index matching policy index patter with both auto_manage disabled $ http PUT :9200/sample-idx-003 settings:='{ "index": { "plugins.index_state_management.auto_manage": "false", "opendistro.index_state_management.auto_manage": "false" } }' ... empty logs ... # ISM does not manage index sample-idx-003 ```

What is the expected behavior?

Can we know why opendistro still exists, other than the reason mentioned above? Is there any plan to completely deprecate opendistro namespace? Can we somehow alias the opendistro namespace over plugins namespace, so that we don't have to manually set auto_manage to both namespaces. There could be many such checks which try to honor both namespaces and fail.

What is your host/environment?

Do you have any screenshots?

Do you have any additional context?