search

confluentinc / kafka-rest

Confluent REST Proxy for Kafka
https://docs.confluent.io/current/kafka-rest/docs/index.html
Other
44 stars 646 forks source link

Is using kafka's port 9092 although KAFKA_REST_CLIENT_SECURITY_PROTOCOL has value SSL #436

Open hectorgarzon opened 6 years ago

hectorgarzon commented 6 years ago

Hi, I have deployed a Kafka cluster v0.10 with endpoints for SSL and PLAINTEXT. Also, I have deployed a component Kafka-rest with the following configuration:

          - name: KAFKA_REST_ZOOKEEPER_CONNECT
            value: "deheremap7213.emea.adsint.biz:2181"
          - name: KAFKA_REST_BOOTSTRAP_SERVERS
            value: "SSL://machine1:9093"
          - name: KAFKA_REST_CLIENT_SECURITY_PROTOCOL
            value: "SSL"

When trying to get messages from a topic using kafka topics-ui then on Kafka-rest's log I can see that kafka-rest is using port 9092:

WARN Fetching topic metadata with correlation id 4 for topics [Set(_schemas)] from broker 
[BrokerEndPoint(3,kafka3,9092)] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException

and also that metadata.broker.list is overridden with port 9092: INFO Property metadata.broker.list is overridden to kafka1:9092,kafka2:9092,kafka3:9092 (kafka.utils.VerifiableProperties)

Also if I delete PLAINTEXT endpoint from zNode on Zookeeper {"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT","SSL":"SSL"},"endpoints":["PLAINTEXT://kafka3:9092","SSL://machine1:9093"],"jmx_port":9999,"host":"kafka3","timestamp":"1527154260933","port":9092,"version":4} then Kafka-rest complains that there is no PLAINTEXT endpoint (why it should require PLAINTEXT endpoint anyway?)

Any idea why this is happening?

Thanks

pharman commented 5 years ago

We are having this problem too. No matter what is defined in properties, V1 always connects using PLAINTEXT. Is this something to do with Zookeeper based consumers?

Poke @ijuma :D