search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
68.56k stars 24.35k forks source link

Shutdown stalled unnecessarily #100028

Open henningandersen opened 9 months ago

henningandersen commented 9 months ago

Elasticsearch Version

8.9

Installed Plugins

ESS

Java Version

bundled

OS Version

ESS

Problem Description

When removing frozen (and cold) nodes, we report stalled back if a frozen shard is unassigned from the node. This could happen due to other instability and should preferably not break the orchestration flow. Instead, we should permit continuing the operation despite any searchable snapshots that are unassigned, since data is not at risk for those.

Additionally (but perhaps dealt with separately), it might be beneficial to check for whether any such shards are currently either throttled or initializing before reporting stalled. This could be beneficial for both regular indices and searchable snapshots, since if any are throttled or initializing we are likely making progress towards a state of no unassigned shards.

Steps to Reproduce

NA

Logs (if relevant)

No response

elasticsearchmachine commented 9 months ago

Pinging @elastic/es-core-infra (Team:Core/Infra)