ConfigMap for Stargate should be replicated if defined at the cluster level

[jsanda]

What is missing? The user can set the cassandraConfigMapRef property for Stargate to point a to a ConfigMap that contains a cassandra.yaml that Stargate will use. We use this extensively in the e2e tests. Here is an example:

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: test
spec:
  cassandra:
    serverVersion: "4.0.1"
    storageConfig:
      cassandraDataVolumeClaimSpec:
        storageClassName: standard
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
    config:
      jvmOptions:
        heapSize: 512Mi
    resources:
      limits:
        memory: 1024Mi
    networking:
      hostNetwork: true
    datacenters:
      - metadata:
          name: dc1
        k8sContext: kind-k8ssandra-0
        size: 2
      - metadata:
          name: dc2
        k8sContext: kind-k8ssandra-1
        size: 2
  stargate:
    size: 1
    cassandraConfigMapRef:
      name: cassandra-config

With this manifest a Stargate object will be created in both the kind-k8ssandra-0 and kind-k8ssandra-1 clusters. Each will be configured with a volume and volume mount for the ConfigMap.

The ConfigMap needs to be created separately in each target cluster. We do this up front as part of the test setup.

If the ConfigMap reference is defined at the cluster level, then the operator should replicate it. Furthermore, this means that the ConfigMap should be created in the same namespace as the K8ssandraCluster.

This is definitely a post-1.0.0 enhancement and another reason to look into kubefed.

For 1.0.0 though, we should document that that the current design/implementation requires you to create the ConfigMap up front in each of the target clusters.

Why do we need it? It will improve the multi-cluster scenarios and make things less error prone for the user. It will also help prevent configuration drift. For example, I might make an ad hoc change in one k8s cluster and forget or fail to apply it in others. If the change is made in the control plane cluster, then the operator can make sure it is propagated.

Environment

• K8ssandra Operator version:

1.0.0-alpha.3

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: K8OP-124

[burmanm]

The SecretReplicator can do this easily. It doesn't really have that much "Secret" defined in it, it could just as well be any object it replicates.

[jsanda]

True, it could be done with SecretReplicator; however, I think it is worthwhile to take another look at kubefed.