Altinity / clickhouse-operator

Altinity Kubernetes Operator for ClickHouse creates, configures and manages ClickHouse® clusters running on Kubernetes
https://altinity.com
Apache License 2.0
1.91k stars 461 forks source link

Operator 0.24 dropping clickhouse keeper limits and initContainer #1546

Closed alexvzin closed 1 day ago

alexvzin commented 1 day ago

I'm using manifest, that works great with operator 0.23.7 But operator 0.24.0 is dropping keeper limits and initcontainer

apiVersion: clickhouse-keeper.altinity.com/v1
kind: ClickHouseKeeperInstallation
metadata:
  name: clickhouse-keeper
spec:
  configuration:
    clusters:
      - name: "keeper"
        layout:
          replicasCount: 3
    settings:
      logger/level: "trace"
      logger/console: "true"
      listen_host: "0.0.0.0"
      keeper_server/storage_path: /var/lib/clickhouse-keeper
      keeper_server/tcp_port: "2181"
      keeper_server/four_letter_word_white_list: "*"
      keeper_server/coordination_settings/raft_logs_level: "information"
      keeper_server/raft_configuration/server/port: "9444"
      prometheus/endpoint: "/metrics"
      prometheus/port: "7000"
      prometheus/metrics: "true"
      prometheus/events: "true"
      prometheus/asynchronous_metrics: "true"
      prometheus/status_info: "false"
  templates:
    podTemplates:
      - name: clickhouse-keeper-pod
        metadata:
          annotations:
            prometheus.io/scrape: "true"
          labels:
            app: clickhouse-keeper
            what: node
        spec:
          securityContext:
            fsGroup: 101 
            runAsUser: 101 
          affinity:
            podAntiAffinity:
              preferredDuringSchedulingIgnoredDuringExecution:
                - weight: 50
                  podAffinityTerm:
                    labelSelector:
                      matchExpressions:
                        - key: "app"
                          operator: In
                          values:
                            - clickhouse-keeper
                    topologyKey: "kubernetes.io/hostname"
          containers:
            - name: clickhouse-keeper
              imagePullPolicy: IfNotPresent
              image: "altinity/clickhouse-keeper:24.3.5.47.altinitystable"
              resources:
                requests:
                  memory: 100Mi
                  cpu: 100m
                limits:
                  memory: 1000Mi
                  cpu: 1000m
          initContainers:
            - command: ["sh", "<MY COMMAND>"]
              image: <MY IMAGE>
              imagePullPolicy: IfNotPresent
              name: server-id-injector
              resources:
                requests:
                  memory: 100Mi
                  cpu: 100m
                limits:
                  memory: 200Mi
                  cpu: 200m

    volumeClaimTemplates:
      - name: both-paths
        spec:
          storageClassName: your-storageclass
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 3Gi

But in StatefulSet I can't see limits and initContainer

..........................................
    spec:
      containers:
      - env:
        - name: CLICKHOUSE_DATA_DIR
          value: /var/lib/clickhouse-keeper
        image: clickhouse/clickhouse-keeper:latest
        imagePullPolicy: Always
        livenessProbe:
          exec:
            command:
            - bash
            - -xc
            - date && OK=$(exec 3<>/dev/tcp/127.0.0.1/2181; printf 'ruok' >&3; IFS=;
              tee <&3; exec 3<&-;);if [[ "${OK}" == "imok" ]]; then exit 0; else exit
              1; fi
          failureThreshold: 10
          initialDelaySeconds: 60
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        name: clickhouse-keeper
        ports:
        - containerPort: 2181
          name: zk
          protocol: TCP
        - containerPort: 9444
          name: raft
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /ready
            port: 9182
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/clickhouse-keeper/keeper_config.d/
          name: chk-clickhouse-keeper-common-configd
        - mountPath: /etc/clickhouse-keeper/users.d/
          name: chk-clickhouse-keeper-common-usersd
        - mountPath: /etc/clickhouse-keeper/conf.d/
          name: chk-clickhouse-keeper-deploy-confd-keeper-0-0
      dnsPolicy: ClusterFirst
      hostAliases:
      - hostnames:
        - chk-clickhouse-keeper-keeper-0-0
        ip: 127.0.0.1
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 420
          name: chk-clickhouse-keeper-common-configd
        name: chk-clickhouse-keeper-common-configd
      - configMap:
          defaultMode: 420
          name: chk-clickhouse-keeper-common-usersd
        name: chk-clickhouse-keeper-common-usersd
      - configMap:
          defaultMode: 420
          name: chk-clickhouse-keeper-deploy-confd-keeper-0-0
        name: chk-clickhouse-keeper-deploy-confd-keeper-0-0
Slach commented 22 hours ago

could you suggest, what was a root cause and which workaround?