search

k8ssandra / cass-operator

The DataStax Kubernetes Operator for Apache Cassandra
https://docs.datastax.com/en/cass-operator/doc/cass-operator/cassOperatorGettingStarted.html
Apache License 2.0
189 stars 66 forks source link

Crossed IPs handling #130

Open jdonenine opened 3 years ago

jdonenine commented 3 years ago

Originally reported in https://github.com/datastax/cass-operator/issues/312 by cscetbon

Sometimes IPs from different cassandra nodes of the same cluster can be exchanged and in that case Cassandra refuses to start. CassKop introduced a fix to this issue described here.

What it does is store the host/ID map and when it sees that the map changed with an exchange of IPs from the previous time it collected it, it kills those pods and let k8s restart them. The current code is available at https://github.com/Orange-OpenSource/casskop/blob/v1.0.0-release/pkg/controller/cassandracluster/reconcile.go#L708-L721 and https://github.com/Orange-OpenSource/casskop/blob/v1.0.0-release/pkg/controller/cassandracluster/reconcile.go#L751-L794).

The goal of this ticket is have it handled in cass-operator.

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: CASS-59

bradfordcp commented 2 years ago

I'm curious what the exact step by step scenario is here to reproduce. I know we've been a bit fuzzy around what happens in this scenario and want to get this corner case covered.