k8ssandra / k8ssandra-operator

The Kubernetes operator for K8ssandra
https://k8ssandra.io/
Apache License 2.0
170 stars 78 forks source link

SchemaDisagreementOnStargate test is flaky #930

Open burmanm opened 1 year ago

burmanm commented 1 year ago

What happened? --- FAIL: TestK8ssandraCluster (291.64s) --- FAIL: TestK8ssandraCluster/AddDcToExistingCluster (56.57s) --- FAIL: TestK8ssandraCluster/AddDcToExistingCluster/SchemaDisagreementOnStargate (2.71s) add_dc_test.go:104: wait for the CassandraInitialized condition to be set add_dc_test.go:731: add Stargate and Reaper add_dc_test.go:573: check that stargate sg1 is created add_dc_test.go:576: update stargate sg1 status to ready add_dc_test.go:588: check that reaper reaper1 is created add_dc_test.go:591: update reaper reaper1 status to ready add_dc_test.go:747: add dc2 to cluster add_dc_test.go:772: Error Trace: /home/runner/work/k8ssandra-operator/k8ssandra-operator/controllers/k8ssandra/add_dc_test.go:772 /home/runner/work/k8ssandra-operator/k8ssandra-operator/controllers/k8ssandra/add_dc_test.go:597 /home/runner/work/k8ssandra-operator/k8ssandra-operator/controllers/k8ssandra/add_dc_test.go:232 Error: Received unexpected error: Operation cannot be fulfilled on k8ssandraclusters.k8ssandra.io "add-dc-test": the object has been modified; please apply your changes to the latest version and try again Test: TestK8ssandraCluster/AddDcToExistingCluster/SchemaDisagreementOnStargate Messages: failed to add dc to K8ssandraCluster FAIL

Did you expect to see something different? This indicates a timing failure or a bug in which Updates are done in incorrect order. Either way, something to investigate and fix.

How to reproduce it (as minimally and precisely as possible):

Environment

insert manifests relevant to the issue
insert K8ssandra Operator logs relevant to the issue here

Anything else we need to know?:

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: K8OP-237

Miles-Garnsey commented 1 year ago

@burmanm assigning you, could this be a good target for the next refactoring week?