linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.73k stars 797 forks source link

add an example for deploying in kubernetes with the strimzi operator #809

Open dcharbonnier opened 6 months ago

dcharbonnier commented 6 months ago

This is a minimal example, the deployment should be tuned.

apiVersion: v1
kind: ConfigMap
metadata:
  name: burrow-config
data:
  burrow.toml: |
    [zookeeper]
    servers=[ "{{cluster}}-cluster-zookeeper-client.{{namespace}}.svc.cluster.local:2181" ]
    timeout=30
    root-path="/tmp/zh"
    tls="zookeeper"

    [tls.zookeeper]
    keyfile="/burrow-ca-key/ca.key"
    certfile="/burrow-ca-crt/ca.crt"
    cafile="/burrow-ca-crt/ca.crt"

    [client-profile.default]
    kafka-version="2.0.1"
    client-id="borrow-client"

    [cluster.local]
    client-profile="default"
    class-name="kafka"
    servers=[ "{{cluster}}-cluster-kafka-brokers.{{namespace}}.svc.cluster.local:9092" ]
    topic-refresh=60
    offset-refresh=30
    groups-reaper-refresh=30

    [httpserver.default]
    address=":8000"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: burrow
  labels:
    k8s-app: burrow
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: burrow
  template:
    metadata:
      labels:
        k8s-app: burrow
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8000"
    spec:
      imagePullSecrets:
        - name: {{imagePullSecret}}
      containers:
        - name: burrow
          image: {{registry}}burrow:{{version}}
          ports:
            - name: api
              containerPort: 8000
          readinessProbe:
            httpGet:
              path: /burrow/admin
              port: 8000
          livenessProbe:
            httpGet:
              path: /burrow/admin
              port: 8000
          resources:
            limits:
              cpu: 100m
              memory: 256M
            requests:
              cpu: 10m
              memory: 10M
          volumeMounts:
            - name: config
              mountPath: /etc/burrow/
            - name: tmp
              mountPath: /tmp
            - name: ca-key
              mountPath: /burrow-ca-key
            - name: ca-crt
              mountPath: /burrow-ca-crt
      volumes:
        - name: config
          configMap:
            name: burrow-config
        - name: ca-key
          secret:
            secretName: {{cluster}}-cluster-cluster-ca
        - name: ca-crt
          secret:
            secretName: {{cluster}}-cluster-cluster-ca-cert
        - name: tmp
          emptyDir:
            sizeLimit: 10Mi
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: burrow-network-policy-zookeeper
spec:
  ingress:
    - from:
        - podSelector:
            matchLabels:
              k8s-app: burrow
      ports:
        - port: 2181
          protocol: TCP
  podSelector:
    matchLabels:
      strimzi.io/cluster: {{cluster}}-cluster
      strimzi.io/kind: Kafka
      strimzi.io/name: {{cluster}}-cluster-zookeeper
  policyTypes:
    - Ingress
abdelrahman-a-ibrahim commented 6 months ago

Hi @dcharbonnier,

Thank you for sharing this with us, I'd like to ask you about the used image for Burrow. Would you please share it?

dcharbonnier commented 5 months ago

I did a clone of burrow on our private git and added ci to build and push an image on our private registry using the included dockerfile