Facilitate the restore of red indices

[renshuki]

Kibana version: 8.3.2 Related issues:

• https://github.com/elastic/kibana/issues/137373
• https://github.com/elastic/kibana/issues/137432

Synopsis

Currently, restoring red indices from a snapshot require the following steps in Kibana UI:

1) Close all the red indices

• Go to Stack Management > Index Management
• Toggle "Include hidden indices"
• Sort by Health column or search for: health:red
• At the bottom of the page select: Rows per page: 100 if you have a high number of red indices that need to be closed
• Select all red indices checkboxes
• Click on Manage x indices > Close indices
• Save the list of red indices somewhere (i.e. text editor). This is a manual process which require to copy-paste indices from the Index Management or the usage of APIs in Kibana Dev Tools (e.g GET _cat/indices?health=red&h=index)

(You may need to repeat the operation if you have more than 100 red indices as there is a limit of 100 rows per page)

2) Restore the closed red indices

• Go to Stack Mangement > Snapshot and Restore
• Select the latest successful snapshot (State: Snapshot Complete) from the list (i.e Snapshot with most recent Date Created and no Failed Shards)
• Click Restore
• Toggle All data streams and indices
• Click Select data streams and indices
• Select the red indices from the dropdown (by cross-checking from the list you saved in 1. which can be very time-consuming and prone to errors)
• Toggle off Restore aliases (as often not needed for red indices recovery scenarios)
• Next, Next and Restore snapshot

Describe the feature

Add a feature to automate and streamline the restore of red indices in a cluster.

Proposals

Below, 2 different alternatives that could be implemented as a feature:

1. Only restore red indices check box from the Restore UI

2. Restore from Index Management

Filter and select the desired indices. Then, Restore indices should prompt a message which allow the users to select from which repository / snapshot they want to restore from and populate the restore UI accordingly (with the red indices pre-selected).

In both proposals (1) and (2), the step of closing red indices to restore should be tranparent for the user - i.e operated in the background before the restore happens.

I'm open to discuss other solutions for this enhancement in order to make the restore process of red indices more user-friendly.

---

CC: @romain-chanu who participated in the brainstorming :heart:

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[romain-chanu]

Also related to https://github.com/elastic/elasticsearch/issues/86606

[elasticmachine]

Pinging @elastic/kibana-management (Team:Kibana Management)