Members unable to join when schedules in different nodes - Githubissues

oracle / mysql-operator

Create, operate and scale self-healing MySQL clusters in Kubernetes

870 stars 235 forks source link

Members unable to join when schedules in different nodes #295

Closed figassis closed 4 years ago

figassis commented 4 years ago

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

MySQL Operator Version: 0.3.0

Environment:

Kubernetes version (use kubectl version): 1.15.0
Cloud provider or hardware configuration: Hetzner Root Server
OS (e.g. from /etc/os-release): Ubutnu 18.04.3 LTS (Bionic Beaver)
Kernel (e.g. uname -a):Linux node1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Others:

What happened?

The first cluster member comes up fine, but the second gets stuck as below. mysql-mysql-cluster-1 1/2 Running

with the mysql-agent container stuck at "Cluster not yet present. Waiting..."

What you expected to happen?

All 3 members to come up healthy

How to reproduce it (as minimally and precisely as possible)?

Failed install

Add mysql="true" label to node1 and node2
Add nodeSelector mysql="true" to the cluster spec
Install mysql-operator in its own namespace, via helm chart
Install mysql-agent ServiceAccount and RoleBinding in "mysql" namespace
Install mysqlcluster helm chart with 3 replicas in "mysql" namespace
Only first member comes up healthy

Successful install

Uninstall cluster
Remove label from node1
Install cluster
All members come up healthy

figassis commented 4 years ago

Issue is related to https://github.com/projectcalico/calico/issues/1709.