scylladb / scylla-operator

The Kubernetes Operator for ScyllaDB
https://operator.docs.scylladb.com/
Apache License 2.0
340 stars 175 forks source link

Flake - Node Setup should propagate mount controller's degraded condition for mount unit configured with a device with an invalid filesystem type [Serial] #2163

Open zimnx opened 1 month ago

zimnx commented 1 month ago

Link to the job that flaked.

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/2160/pull-scylla-operator-master-e2e-gke-serial/1849096276826656768

Snippet of what failed.

 Node Setup should propagate mount controller's degraded condition for mount unit configured with a device with an invalid filesystem type [Serial]
github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig/nodeconfig_disksetup.go:367
  STEP: Creating a new namespace @ 10/23/24 14:49:35.193
  Oct 23 14:49:35.205: INFO: Created namespace "e2e-test-nodesetup-77rln-0-zk5c2".
  STEP: Waiting for service account token Secret "e2e-user-token" in namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:49:35.228
I1023 14:49:35.235196       1 cache/reflector.go:351] Caches populated for *v1.Secret from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
  STEP: Waiting for default ServiceAccount in namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:49:35.297
I1023 14:49:35.302799       1 cache/reflector.go:351] Caches populated for *v1.ServiceAccount from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
  STEP: Waiting for kube-root-ca.crt in namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:49:35.305
I1023 14:49:35.309867       1 cache/reflector.go:351] Caches populated for *v1.ConfigMap from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
  STEP: Verifying there is at least one scylla node @ 10/23/24 14:49:35.311
  Oct 23 14:49:35.318: INFO: There are 1 scylla nodes
  STEP: Snapshotting object scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster" @ 10/23/24 14:49:35.318
  STEP: Deleting object scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster". @ 10/23/24 14:49:35.322
I1023 14:49:35.322759       1 framework/cleanup.go:102] "No existing object found" GVR="scylla.scylladb.com/v1alpha1, Resource=nodeconfigs" Instance="cluster"
  STEP: Creating NodeConfig @ 10/23/24 14:49:35.327
  STEP: Waiting for NodeConfig to be in a degraded state @ 10/23/24 14:49:35.336
I1023 14:49:35.340960       1 cache/reflector.go:351] Caches populated for *v1alpha1.NodeConfig from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
  [FAILED] in [It] - github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig/nodeconfig_disksetup.go:317 @ 10/23/24 14:54:35.337
  STEP: Collecting events from namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:54:35.337
  STEP: Found 0 events. @ 10/23/24 14:54:35.343
  STEP: Collecting dumps from namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:54:35.343
  STEP: Collecting global scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster" for namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:54:35.628
  STEP: Destroying namespace "e2e-test-nodesetup-77rln-0-zk5c2". @ 10/23/24 14:54:35.639
  STEP: Waiting for namespace "e2e-test-nodesetup-77rln-0-zk5c2" to be removed. @ 10/23/24 14:54:35.657
I1023 14:54:35.670995       1 cache/reflector.go:351] Caches populated for *unstructured.Unstructured from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
I1023 14:54:41.434624       1 framework/cleanup.go:73] "Namespace removed." Namespace="e2e-test-nodesetup-77rln-0-zk5c2"
  STEP: Deleting object scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster". @ 10/23/24 14:54:41.434
  STEP: Waiting for object scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster" to be removed. @ 10/23/24 14:54:41.451
I1023 14:54:41.459549       1 cache/reflector.go:351] Caches populated for *unstructured.Unstructured from k8s.io/client-go@v0.29.8/tools/cache/reflector.go:229
  STEP: Object scylla.scylladb.com/v1alpha1, Resource=nodeconfigs "cluster" has been removed. @ 10/23/24 14:54:42.274
• [FAILED] [307.081 seconds]
Node Setup should propagate mount controller's degraded condition [It] for mount unit configured with a device with an invalid filesystem type [Serial]
github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig/nodeconfig_disksetup.go:367
  [FAILED] Unexpected error:
      <*fmt.wrapError | 0xc00039c000>: 
      waiting has been interupted ([true,false]): timed out waiting for the condition
      {
          msg: "waiting has been interupted ([true,false]): timed out waiting for the condition",
          err: <wait.errInterrupted>{
              cause: <*errors.errorString | 0xc0001336d0>{
                  s: "timed out waiting for the condition",
              },
          },
      }
  occurred
  In [It] at: github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig/nodeconfig_disksetup.go:317 @ 10/23/24 14:54:35.337
  Full Stack Trace
    github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig.init.func1.3({0x3594ae8, 0xc000d0d170}, 0xc00034a710)
        github.com/scylladb/scylla-operator/test/e2e/set/nodeconfig/nodeconfig_disksetup.go:317 +0x6d9
    reflect.Value.call({0x1eec8a0?, 0xc0004733e0?, 0x46bc1d?}, {0x23cbcf7, 0x4}, {0xc000d0d200, 0x2, 0xc000d0d200?})
        reflect/value.go:581 +0xca6
    reflect.Value.Call({0x1eec8a0?, 0xc0004733e0?, 0x35ad2b0?}, {0xc000d0d200?, 0x0?, 0xc000d06820?})
        reflect/value.go:365 +0xb9
------------------------------