Update _cat/indices to new, paginated _list/indices API

[amitgalitz]

Is your feature request related to a problem? We should use the new _list/indices API instead of _cat/indices to better handle fetching of indices as current cat API consumes too many resources.

Issue for new _list/indices API: https://github.com/opensearch-project/OpenSearch/pull/15986 issue for context on limitation of CAT api: https://github.com/opensearch-project/OpenSearch/issues/15954

What solution would you like? Changing to a paginated API means we need to potentially change our UI as well as we will either need a mechanism to make API calls to the next set of results or relay more so on our search functionality and only display a limited (1k) number of results on the frontend at a time.

Options:

Option 1: Make the changes to list/indices and we set default to 1k without implementing pagination.

Pros:

• Quicker implementation and clusters with over 1k indices are less common
• Users don’t usually scroll through 1k indices in drop down without search

Cons:

• Users will only see up to 1k indices, They need to search to easily find their index if they have more then 1k but technically it also means search results will only show up to 1k results.

Option 2: Change index selection drop down to include pagination or other UI based mechanism to fetch the next page of results

Pros:

• This means that the user has a way to view all indices through the drop down

Cons:

• Viewing all indices through a drop down when we reach 10k+ indices might be less useful as the number of "pages" might get too high

[kamingleung]

I would recommend providing a drop down that can lazy load. Specifically, it will load the next batch of data once the user scrolls to the end of first batch. @virajsanghvi, does selectable or combobox OUI components support any sort of lazy loading?

[amitgalitz]

Additionally @virajsanghvi I see that we have an example of a combo box utilizing react-window but I am not sure if the demo JS is correct: https://oui.opensearch.org/1.17/#/forms/combo-box#virtualized

Would this solution be good to use here for the paginated _list/indices API or should we have another lazy loading solution when we reach the bottom of the list.

[virajsanghvi]

I think the combo box virtualizes by default, which is why the demo js doesn't have anything called out. Virtualization assumes you have all the options though.

It looks like the component supports async search, https://oui.opensearch.org/1.17/#/forms/combo-box#async, but that doesn't handle loading.

I don't see a component that supports this out of the box, but maybe @AMoo-Miki may know of a component or example somewhere.

[andrross]

[Catch All Triage - 1, 2, 3, 4, 5]