Proxy closing connection with Kafka permaturely

Hello!

I'm trying to set-up kafka-proxy in Kubernetes to connect to Kafka outside Kubernetes. I am able to connect and use Kafka as expected (I'm using edenhill's kcat for this), but after roughly 2 minutes the proxy just closes the connection to the broker without further explanation.

This doesn't happen when running the proxy in docker, only in Kubernetes, which suggests a problem when dealing with Kubernetes networking. Does anyone know what is happening or why? Thanks in advance!

Bellow are my manifests for Deployment and Service, and the logs at debug level. The BROKER-IP used is censored, but it does work and resolve into a public IP where the broker is.

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
   name: myapp
spec:
  selector:
    matchLabels:
      app: myapp
  replicas: 1
  template:
    metadata:
      labels:
        app: myapp
    spec:
        - name: kafka-proxy
          image: grepplabs/kafka-proxy:v0.3.1
          args:
            - 'server'
            - '--bootstrap-server-mapping=BROKER-IP:9095,0.0.0.0:3000,$(MY_NODE_IP):30001'
            - '--debug-enable'
            - '--log-level=debug'
            - '--dynamic-listeners-disable'
            - '--debug-listen-address=0.0.0.0:6060'
          env:
          - name: MY_NODE_IP
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP

NodePort Service

apiVersion: v1
kind: Service
metadata:
  name: my-service
  labels:
    app: myapp
spec:
  type: NodePort
  selector:
    app: myapp
  ports:
    - name: normal
      port: 3000
      targetPort: 3000
      nodePort: 30001

Error message

time="2022-09-20T18:52:06Z" level=info msg="Starting kafka-proxy version v0.3.1"
time="2022-09-20T18:52:06Z" level=info msg="Bootstrap server BROKER-IP:9095 advertised as 10.11.19.221:30001"
time="2022-09-20T18:52:06Z" level=info msg="Listening on 0.0.0.0:3000 ([::]:3000) for remote BROKER-IP::9095"
time="2022-09-20T18:52:06Z" level=info msg="Ready for new connections"
time="2022-09-20T18:53:35Z" level=info msg="Closing connections"
time="2022-09-20T18:53:35Z" level=info msg="Proxy is stopped"
time="2022-09-20T18:53:35Z" level=info msg="Exit received signal terminated"

grepplabs / kafka-proxy

Proxy closing connection with Kafka permaturely #110