adwk67
commented
2 weeks ago Below is a draft custom resource for a Kafka cluster, using overrides. a few things to note:
- the custom image
docker.stackable.tech/apoc/stackable/kafka:3.7.1-stackable0.0.0-cyrus-sasl-gssapiadds(microdnf install) cyrus-sasl-gssapito the kafka image - the kcat readines probe command below times out.
/stackable/kcat -Vcan be used if we just want the pods to come up, but then the full command can be successfully executed when shelling into the kcat container! - the
argsneeds to be overridden to be able to changelistener.security.protocol.map - running a similar kcat command from a client (e.g. a Job) allows us to fetch metadata, and to create topics, ~but consuming from a topic results in the following error:
Server kafka/172.19.0.5@CLUSTER.LOCAL not found in Kerberos database, i.e. the advertised listener needs to be kerberos-ized as well.~- Update: this works if the advertised listener is changed to
kafka.stackable-products.svc.cluster.local:9093as shown below - Update 2: service replaced with
scope: podfor multiple brokers
- Update: this works if the advertised listener is changed to
---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperZnode
metadata:
name: kafka-znode
namespace: stackable-products
spec:
clusterRef:
name: zookeeper
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: kafka
namespace: stackable-products
spec:
image:
productVersion: 3.7.1
repo: docker.stackable.tech/apoc/stackable
pullPolicy: IfNotPresent
clusterConfig:
tls:
serverSecretClass: tls
zookeeperConfigMapName: kafka-znode
brokers:
config:
logging:
enableVectorAgent: False
resources:
memory:
limit: '1.3Gi'
roleGroups:
default:
replicas: 1
envOverrides:
KRB5_CONFIG: "/etc/krb5.conf"
configOverrides:
server.properties:
sasl.enabled.mechanisms: "GSSAPI"
sasl.kerberos.service.name : "kafka"
sasl.mechanism.inter.broker.protocol: "GSSAPI"
podOverrides:
spec:
containers:
- name: kafka
volumeMounts:
- name: kerberos
mountPath: /stackable/kerberos
- name: kerberos
mountPath: /etc/krb5.conf
subPath: krb5.conf
args:
- |2
prepare_signal_handlers()
{
unset term_child_pid
unset term_kill_needed
trap 'handle_term_signal' TERM
}
handle_term_signal()
{
if [ "${term_child_pid}" ]; then
kill -TERM "${term_child_pid}" 2>/dev/null
else
term_kill_needed="yes"
fi
}
wait_for_termination()
{
set +e
term_child_pid=$1
if [[ -v term_kill_needed ]]; then
kill -TERM "${term_child_pid}" 2>/dev/null
fi
wait ${term_child_pid} 2>/dev/null
trap - TERM
wait ${term_child_pid} 2>/dev/null
set -e
}
rm -f /stackable/log/_vector/shutdown
prepare_signal_handlers
bin/kafka-server-start.sh \
/stackable/config/server.properties \
--override "zookeeper.connect=$ZOOKEEPER" \
--override "listeners=CLIENT://0.0.0.0:9093,INTERNAL://0.0.0.0:19093" \
--override "advertised.listeners=CLIENT://$POD_NAME.kafka-broker-default.stackable-products.svc.cluster.local:9093,INTERNAL://$POD_NAME.kafka-broker-default.stackable-products.svc.cluster.local:19093" \
--override "listener.security.protocol.map=CLIENT:SASL_SSL,INTERNAL:SSL" \
--override "listener.name.client.gssapi.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab=\"/stackable/kerberos/keytab\" principal=\"kafka/$POD_NAME.kafka-broker-default.stackable-products.svc.cluster.local@CLUSTER.LOCAL\";" &
wait_for_termination $!
mkdir -p /stackable/log/_vector && touch /stackable/log/_vector/shutdown
command:
- /bin/bash
- -x
- -euo
- pipefail
- -c
- name: kcat-prober
env:
- name: KRB5_CONFIG
value: /etc/krb5.conf
volumeMounts:
- name: kerberos
mountPath: /stackable/kerberos
- name: kerberos
mountPath: /etc/krb5.conf
subPath: krb5.conf
readinessProbe:
exec:
command:
- /stackable/kcat
- -V
volumes:
- name: kerberos
ephemeral:
volumeClaimTemplate:
metadata:
annotations:
secrets.stackable.tech/class: kerberos
secrets.stackable.tech/scope: pod
secrets.stackable.tech/kerberos.service.names: kafka
spec:
storageClassName: secrets.stackable.tech
accessModes:
- ReadWriteOnce
resources:
requests:
storage: "1"
Currently we only support mutual tls as authentication mechanism, especially in existing installations, Kerberos may be a preferred way of doing this.