search

strimzi / strimzi-kafka-operator

Apache Kafka® running on Kubernetes
https://strimzi.io/
Apache License 2.0
4.86k stars 1.3k forks source link

Fail to access Kafka brokers outside of k8s clusters if using loadbalancer #1414

Closed mathspanda closed 5 years ago

mathspanda commented 5 years ago

This is my Kafka CR.

apiVersion: kafka.strimzi.io/v1alpha1
kind: Kafka
metadata:
  name: link
spec:
  kafka:
    version: 2.1.0
    replicas: 3
    listeners:
      plain: {}
      tls: {}
      external:
        type: nodeport
        overrides:
          bootstrap:
            nodePort: 32100
          brokers:
          - broker: 0
            nodePort: 32000
          - broker: 1
            nodePort: 32001
          - broker: 2
            nodePort: 32002
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      log.message.format.version: "2.1"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

This is the result of kubectl get all -n linpeiyu.

root@office-k8s-master:~# kubectl -n linpeiyu get all
NAME                                             READY     STATUS    RESTARTS   AGE
pod/link-entity-operator-59759f5944-ndfpq        3/3       Running   0          46m
pod/link-kafka-0                                 2/2       Running   1          46m
pod/link-kafka-1                                 2/2       Running   0          46m
pod/link-kafka-2                                 2/2       Running   0          46m
pod/link-zookeeper-0                             2/2       Running   0          47m
pod/link-zookeeper-1                             2/2       Running   0          47m
pod/link-zookeeper-2                             2/2       Running   0          47m
pod/silly-sloth-kafka-manager-7c9cdfccbb-vxwtt   1/1       Running   1          3d
pod/strimzi-cluster-operator-6847b87c98-xxx9f    1/1       Running   1          3d

NAME                                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/link-kafka-0                    NodePort    172.22.235.152   <none>        9094:32000/TCP               46m
service/link-kafka-1                    NodePort    172.22.202.38    <none>        9094:32001/TCP               46m
service/link-kafka-2                    NodePort    172.22.44.60     <none>        9094:32002/TCP               46m
service/link-kafka-bootstrap            ClusterIP   172.22.59.170    <none>        9091/TCP,9092/TCP,9093/TCP   46m
service/link-kafka-brokers              ClusterIP   None             <none>        9091/TCP,9092/TCP,9093/TCP   46m
service/link-kafka-external-bootstrap   NodePort    172.22.158.26    <none>        9094:32100/TCP               46m
service/link-zookeeper-client           ClusterIP   172.22.115.82    <none>        2181/TCP                     47m
service/link-zookeeper-nodes            ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   47m
service/silly-sloth-kafka-manager       NodePort    172.22.181.199   <none>        9000:31900/TCP               7d

NAME                                        DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/link-entity-operator        1         1         1            1           46m
deployment.apps/silly-sloth-kafka-manager   1         1         1            1           7d
deployment.apps/strimzi-cluster-operator    1         1         1            1           7d

NAME                                                   DESIRED   CURRENT   READY     AGE
replicaset.apps/link-entity-operator-59759f5944        1         1         1         46m
replicaset.apps/silly-sloth-kafka-manager-7c9cdfccbb   1         1         1         7d
replicaset.apps/strimzi-cluster-operator-6847b87c98    1         1         1         7d

NAME                              DESIRED   CURRENT   AGE
statefulset.apps/link-kafka       3         3         46m
statefulset.apps/link-zookeeper   3         3         47m

Failed to send messages through command kafka-console-producer.sh. 100.100.62.195 is the host ip.

root@office-k8s-master:~/linpeiyu/kafka_2.11-2.1.1/bin# ./kafka-console-producer.sh --broker-list 100.100.62.195:32100 --topic test-topic
>1
[2019-03-07 08:30:45,626] ERROR Error when sending message to topic test-topic with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
>

Is there something wrong?

mathspanda commented 5 years ago

/cc @scholzj

scholzj commented 5 years ago

For some historical reasons, the nodeport and loadbalancer listeners are TLS by default. And your producer doesn't seem to have any TLS configuration. You can disable the TLS by adding tls: false. E.g.:

      # ...
      external:
        type: nodeport
        tls: false
        overrides:
          bootstrap:
            nodePort: 32100
          brokers:
          - broker: 0
            nodePort: 32000
          - broker: 1
            nodePort: 32001
          - broker: 2
            nodePort: 32002
      # ...
mathspanda commented 5 years ago

It works. By the way, is there any way to expose zookeeper cluster outside? Some of kafka commands like kafka-topics.sh still need to specify address of zookeeper.

scholzj commented 5 years ago

We don't really support accessing Zookeeper even from 3rd party applications inside OpenShift. The reason is that Zookeeper is hard to secure etc.

If you really want, you can use this: https://gist.github.com/scholzj/6cfcf9f63f73b54eaebf60738cfdbfae

It should be quite simple to change the service to use node ports as well if you need it outside. But it is very insecure.