k8ssandra / k8ssandra-operator

The Kubernetes operator for K8ssandra
https://k8ssandra.io/
Apache License 2.0
168 stars 78 forks source link

Unable to start due to PVC error from metrics-agent-config #1225

Open gerbal opened 7 months ago

gerbal commented 7 months ago

What happened?

We are attempting to start a k8ssandra-operator cluster but are running into undocumented errors when the stateful set is attempting to mount the metrics-agent-config PVC

 statefulset-controller  create Pod cassandra-dc2-default-sts-0 in StatefulSet cassandra-dc2-default-sts failed error: failed to create PVC metrics-agent-config-cassandra-dc2-default-sts-0: PersistentVolumeClaim "metrics-agent-config-cassandra-dc2-default-sts-0" is invalid: [spec.accessModes: Required value: at least 1 access mode is required, spec.resources[storage]: Required value]

Did you expect to see something different?

I would expect the metrics-agent-config to be mounted by default from the configmap.

How to reproduce it (as minimally and precisely as possible):

Environment

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: cassandra
spec:
  auth: false

  # External datacenters, points to other kassandra instance
  externalDatacenters:
    - "dc1"

  cassandra:
    serverVersion: "3.11.16"
    additionalSeeds:
      - "127.0.0.224"
      - "127.0.3.132"
      - "127.0.2.146"
    containers:
      - name: cassandra
        securityContext:
          readOnlyRootFilesystem: false
          capabilities:
            add: ["IPC_LOCK"]
    datacenters:
      - metadata:
          name: dc2
        size: 3
        config:
          jvmOptions:
            heapSize: 12Gi
            heapNewGenSize: 2Gi
        resources:
          requests:
            cpu: 2
            memory: 24Gi
        storageConfig:
          cassandraDataVolumeClaimSpec:
            storageClassName: ssd
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 200Gi

see attached file k8ssandra-operator-logs.csv

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: K8OP-42

burmanm commented 7 months ago

We do not create a PVC for metrics-agent-config (it's mounted as ConfigMap), so this error is weird. Did the cluster have some old version of CassandraDatacenter CRD and it for some reason was not updated to k8ssandra-operator v1.13.0 one? For example, did it have some older version of cass-operator running at some point.

How does the StatefulSet output look like?