Elasendra Error - shard or keyspace unavailable

[sujeetcityfalcon]

We have three node elasendra cluster.We upgraded it recently, but after that I can see indices on each node are red. cluster state is showing error like: shard or keyspace unavailable . We tried to delete indices but it got hung and executing from last 30 min, so canceled it.

Elasendra version: 6.8.4.16

Errors in log: [f_historical_stories_qa_2020_03_30][0], node[0fObMI6wRVqgLlsNtKHeCA], [P], recovery_source[local shards recovery], s[UNASSIGNED], unassigned_info[[reason=ALLOCATION_FAILED], at[1970-01-01T00:00:00.000Z], failed_attempts[1], delayed=false, details[shard or keyspace unavailable], allocation_status[deciders_no]]

curl -XGET 'http://localhost:9200/_cluster/state/?pretty=true'

{
          "state" : "UNASSIGNED",
          "primary" : true,
          "node" : "2c19J9KBSlWuOA_iSwufGw",
          "relocating_node" : null,
          "shard" : 0,
          "index" : "h_historical_stories_qa_2022_04_22",
          "token_ranges" : [
            "(-9223372036854775808,9142068006140963677]"
          ],
          "recovery_source" : {
            "type" : "LOCAL_SHARDS"
          },
          "unassigned_info" : {
            "reason" : "ALLOCATION_FAILED",
            "at" : "1970-01-01T00:00:00.000Z",
            "failed_attempts" : 1,
            "delayed" : false,
            "details" : "shard or keyspace unavailable",
            "allocation_status" : "deciders_no"
          }

Appreciate for your help.

Thanks

[sujeetcityfalcon]

posted under elassandr project.