Then my connection gets redirected to the Kafka broker, and is using an Internal IP address that I cannot access 100.64.0.15:30896 and results in a timeout error. I would like it to redirect to port 30896 over localhost like above.
ℹ kafka-consumer-log {"severity":7,"fac":"BRKMAIN","message":"[thrd:100.64.0.15:30896/0]: 100.64.0.15:30896/0: Enter main broker thread"}
I'm not sure how to adjust this redirect to the broker using the helm chart. Here's my helm chart configuration:
externalAccess:
## @param externalAccess.enabled Enable Kubernetes external cluster access to Kafka brokers
##
enabled: true
## External IPs auto-discovery configuration
## An init container is used to auto-detect LB IPs or node ports by querying the K8s API
## Note: RBAC might be required
##
autoDiscovery:
## @param externalAccess.autoDiscovery.enabled Enable using an init container to auto-detect external IPs/ports by querying the K8s API
##
enabled: true
## Bitnami Kubectl image
## ref: https://hub.docker.com/r/bitnami/kubectl/tags/
## @param externalAccess.autoDiscovery.image.registry Init container auto-discovery image registry
## @param externalAccess.autoDiscovery.image.repository Init container auto-discovery image repository
## @param externalAccess.autoDiscovery.image.tag Init container auto-discovery image tag (immutable tags are recommended)
## @param externalAccess.autoDiscovery.image.digest Kubectl image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag
## @param externalAccess.autoDiscovery.image.pullPolicy Init container auto-discovery image pull policy
## @param externalAccess.autoDiscovery.image.pullSecrets Init container auto-discovery image pull secrets
##
image:
registry: docker.io
repository: bitnami/kubectl
tag: 1.25.8-debian-11-r2
digest: ""
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: https://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets (secrets must be manually created in the namespace)
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
## e.g:
## pullSecrets:
## - myRegistryKeySecretName
##
pullSecrets: []
## Init Container resource requests and limits
## ref: https://kubernetes.io/docs/user-guide/compute-resources/
## @param externalAccess.autoDiscovery.resources.limits The resources limits for the auto-discovery init container
## @param externalAccess.autoDiscovery.resources.requests The requested resources for the auto-discovery init container
##
resources:
limits: {}
requests: {}
## Parameters to configure K8s service(s) used to externally access Kafka brokers
## Note: A new service per broker will be created
##
service:
## @param externalAccess.service.type Kubernetes Service type for external access. It can be NodePort, LoadBalancer or ClusterIP
##
type: NodePort
## @param externalAccess.service.ports.external Kafka port used for external access when service type is LoadBalancer
##
ports:
external: 9094
## @param externalAccess.service.loadBalancerIPs Array of load balancer IPs for each Kafka broker. Length must be the same as replicaCount
## e.g:
## loadBalancerIPs:
## - X.X.X.X
## - Y.Y.Y.Y
##
loadBalancerIPs: []
## @param externalAccess.service.loadBalancerNames Array of load balancer Names for each Kafka broker. Length must be the same as replicaCount
## e.g:
## loadBalancerNames:
## - broker1.external.example.com
## - broker2.external.example.com
##
loadBalancerNames: []
## @param externalAccess.service.loadBalancerAnnotations Array of load balancer annotations for each Kafka broker. Length must be the same as replicaCount
## e.g:
## loadBalancerAnnotations:
## - external-dns.alpha.kubernetes.io/hostname: broker1.external.example.com.
## - external-dns.alpha.kubernetes.io/hostname: broker2.external.example.com.
##
loadBalancerAnnotations: []
## @param externalAccess.service.loadBalancerSourceRanges Address(es) that are allowed when service is LoadBalancer
## ref: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
## e.g:
## loadBalancerSourceRanges:
## - 10.10.10.0/24
##
loadBalancerSourceRanges: []
## @param externalAccess.service.nodePorts Array of node ports used for each Kafka broker. Length must be the same as replicaCount
## e.g:
## nodePorts:
## - 30001
## - 30002
##
nodePorts: [32487]
## @param externalAccess.service.externalIPs Use distinct service host IPs to configure Kafka external listener when service type is NodePort. Length must be the same as replicaCount
## e.g:
## externalIPs:
## - X.X.X.X
## - Y.Y.Y.Y
##
externalIPs: ["100.64.0.15"]
## @param externalAccess.service.useHostIPs Use service host IPs to configure Kafka external listener when service type is NodePort
##
useHostIPs: false
## @param externalAccess.service.usePodIPs using the MY_POD_IP address for external access.
##
usePodIPs: false
## @param externalAccess.service.domain Domain or external ip used to configure Kafka external listener when service type is NodePort or ClusterIP
## NodePort: If not specified, the container will try to get the kubernetes node external IP
## ClusterIP: Must be specified, ingress IP or domain where tcp for external ports is configured
##
domain: "kafka.kube"
## @param externalAccess.service.publishNotReadyAddresses Indicates that any agent which deals with endpoints for this Service should disregard any indications of ready/not-ready
## ref: https://kubernetes.io/docs/reference/kubernetes-api/service-resources/service-v1/
##
publishNotReadyAddresses: false
## @param externalAccess.service.labels Service labels for external access
##
labels: {}
## @param externalAccess.service.annotations Service annotations for external access
##
annotations: {}
## @param externalAccess.service.extraPorts Extra ports to expose in the Kafka external service
##
extraPorts: []
I am using this Kafka helm chart provided by Bitnami: https://github.com/bitnami/charts/tree/main/bitnami/kafka/
I am accessing Kafka in Kubernetes over a VPN connection using an SSH tunnel.
I am having a problem when Kafka redirects my connection to an internal IP address that I cannot reach over the VPN.
The external listener for Kafka works fine as that first connection is made to localhost:9094 to a port I have opened up in the SSH tunnel.
Then my connection gets redirected to the Kafka broker, and is using an Internal IP address that I cannot access 100.64.0.15:30896 and results in a timeout error. I would like it to redirect to port 30896 over localhost like above.
I'm not sure how to adjust this redirect to the broker using the helm chart. Here's my helm chart configuration: