averemee-si / kafka-k8s

Kubernetes StatefulSet wrapper for Apache Kafka official docker image
https://a2.solutions/
Apache License 2.0
1 stars 0 forks source link

Use the start script over a configmap #1

Open Zetanova opened 7 months ago

Zetanova commented 7 months ago

A better approach is to use the start script https://github.com/averemee-si/kafka-k8s/blob/main/run over a configmap

configmap example

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka
data:
  run: |
    #! /bin/bash

    if [ ! -z "$REPLICAS" ]; then
      HOSTNAME_PATTERN=`echo "${HOSTNAME}" | cut -d'-' -f 1`
      KAFKA_NODE_ID=`echo "${HOSTNAME}" | rev | cut -d'-' -f 1 | rev`
      DOMAIN_NAME=`hostname -d`
      NUM=0
      KAFKA_CONTROLLER_QUORUM_VOTERS=""
      while [ "$NUM" -lt "$REPLICAS" ]; do
        VOTER="${NUM}@${HOSTNAME_PATTERN}-${NUM}.${DOMAIN_NAME}:9093"
        if [ "$NUM" -gt "0" ]; then
          KAFKA_CONTROLLER_QUORUM_VOTERS="$KAFKA_CONTROLLER_QUORUM_VOTERS,"
        fi
        KAFKA_CONTROLLER_QUORUM_VOTERS="${KAFKA_CONTROLLER_QUORUM_VOTERS}${VOTER}"
        NUM=$((NUM+1))
      done
      export KAFKA_NODE_ID KAFKA_CONTROLLER_QUORUM_VOTERS
      echo "Kafka node.id is set to ${KAFKA_NODE_ID}"
      echo "Kafka controller.quorum.voters is set to ${KAFKA_CONTROLLER_QUORUM_VOTERS}"
    fi

    /etc/kafka/docker/run

sts config

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
  labels:
    app: kafka-app
spec:
  serviceName: kafka-headless-svc
  replicas: 3
  selector:
    matchLabels:
      app: kafka-app
  template:
    metadata:
      labels:
        app: kafka-app
    spec:
      containers:
        - name: kafka-container
          image: apache/kafka:latest
          command:
            - "bash"
            - "/mnt/config-map/run"
          ports:
            - containerPort: 9092
            - containerPort: 9093
          env:
            - name: REPLICAS
              value: '3'
            - name: CLUSTER_ID
              value: ryGWk9_ISQ2OxoXJp4NK5Q
            - name: KAFKA_PROCESS_ROLES
              value: 'controller,broker'
            - name: KAFKA_LISTENERS
              value: 'PLAINTEXT://:9092,CONTROLLER://:9093' 
            - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
              value: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT'
            - name: KAFKA_CONTROLLER_LISTENER_NAMES
              value: 'CONTROLLER'
            - name: KAFKA_INTER_BROKER_LISTENER_NAME
              value: 'PLAINTEXT'
            - name: KAFKA_LOG_DIRS
              value: '/tmp/kraft-combined-logs'
          volumeMounts:
            - name: config
              mountPath: /mnt/config-map
      volumes:      
      - name: config
        configMap:
          name: kafka
averemee-si commented 7 months ago

Hi @Zetanova ,

Thanks! Working on it.

Regards, Aleksei