Upgrading from 2.9 to 2.10 occasionally causes a rolling restart of Elasticsearch cluster. - Githubissues

elastic / cloud-on-k8s

Elastic Cloud on Kubernetes

Other

58 stars 708 forks source link

Upgrading from 2.9 to 2.10 occasionally causes a rolling restart of Elasticsearch cluster. #7359

Open BenB196 opened 11 months ago

BenB196 commented 11 months ago

Bug Report

What did you do?

Upgraded from 2.9.0 to 2.10.0

What did you expect to see?

I expected not to see a rolling restart of Elasticsearch clusters based on the docs

What did you see instead? Under which circumstances?

I sometimes saw that the upgrade would cause a rolling restart of Elasticsearch clusters

Environment

ECK version:

2.10.0
Kubernetes information:

insert any information about your Kubernetes environment that could help us:
- On premise Yes
- Cloud: GKE / EKS / AKS N/A
- Kubernetes distribution: Openshift / Rancher / PKS v1.26.10+rke2r2

I'm not sure why, but this didn't happen on all ECK operator upgrades.

Edit: I can provide logs if needed, just need to know what I should look for/provide

ccaillet1974 commented 9 months ago

Same problem from 2.10.0 to 2.11.0 automatic rolling restart of all elasticsearch instances (data nodes, master nodes, ml, transforms ...) but NOT kibana instances.

operator deployed on bare-metal kubernetes (1.27.x and 1.28.x by kubespray).