Cannot access kafka on port 9092

[vishnu-chalil]

Hi, I tried using this for a running kafka connect on docker. But Iam not able to acess kafka from the port 9092. below is the code.

` Properties prop = new Properties();

    prop.setProperty("bootstrap.servers", "host.docker.internal:9092");

    AdminClient admin = AdminClient.create(prop);

    boolean topicExists = admin.listTopics().names().get().stream().anyMatch(topicName -> topicName.equalsIgnoreCase(queryTopic));

    admin.close();
    `

Apart from host.docker.internal:9092 I have tried localhost:9092 and kafka-connect_kafka_1:9092(image name). But no luck. Any thoughts on how to get around this.

[OneCricketeer]

Where is your broker actually running? Why are you writing code to list topics inside a connector?

[vishnu-chalil]

Why are you writing code to list topics inside a connector?

Its for the purpose of checking weather a topic I'am trying to create exists.

Where is your broker actually running?

The broker is running on port 9092 as far as I understood, that's the port that has been exposed too in the docker compose file. The kafka connect is able to communicate with kafka though, I was able to create connectors etc. Not sure why direct access doesn't work.

[OneCricketeer]

Assuming you have a similar compose file as me, you need to use kafka:29092, as listed in the advertised listeners

https://github.com/OneCricketeer/apache-kafka-connect-docker/blob/master/docker-compose.yml#L32

[vishnu-chalil]

kafka:29092 didn't work, but localhost:29092 did work. kafka:29092 was throowing error org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers.

I have changed the docker-compose file and have used different versions of kafka and zookeeper. The whole docker-compose is given below.

`

version: '3'

services: zookeeper: image: bitnami/zookeeper:3.5.7-debian-10-r23 ports:

• '2181:2181' environment:
• ALLOW_ANONYMOUS_LOGIN=yes volumes:

• 'zookeeper_data:/bitnami'

  kafka: image: 'bitnami/kafka:2' ports:

• '9092:9092'
• '29092:29092' volumes:
• 'kafka_data:/bitnami'

• D:/kafka/kafka-connect/data/lipsum.txt:/data/lipsum.txt:ro # Some data to produce

  environment:

• KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
• ALLOW_PLAINTEXT_LISTENER=yes
• KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,PLAINTEXT_HOST://:29092
• KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
• KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092

• KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092

  depends_on:

• zookeeper

  Jib app

  connect-jib: image: cricketeerone/apache-kafka-connect:2.5.0 depends_on:

• kafka ports:

• '8083:8083' environment: CONNECT_REST_ADVERTISED_HOST_NAME: connect-jib CONNECT_BOOTSTRAP_SERVERS: kafka:9092

  CONNECT_GROUP_ID: cg_connect-idea CONNECT_CONFIG_STORAGE_TOPIC: connect-jib_config CONNECT_OFFSET_STORAGE_TOPIC: connect-jib_offsets CONNECT_STATUS_STORAGE_TOPIC: connect-jib_status

  Cannot be higher than the number of brokers in the Kafka cluster

  CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1 CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1 CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1

  CONNECT_KEY_CONVERTER: org.apache.kafka.connect.converters.ByteArrayConverter CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.converters.ByteArrayConverter

  CONNECT_PLUGIN_PATH: /app/lib

  Logging

  Connect client overrides

  CONNECT_TASK_SHUTDOWN_GRACEFUL_TIMEOUT_MS: 30000 CONNECT_OFFSET_FLUSH_INTERVAL_MS: 900000 CONNECT_CONSUMER_MAX_POLL_RECORDS: 500

volumes: zookeeper_data: driver: local kafka_data: driver: local

`

Any thoughts on why kafka:29092 didn't work?

[OneCricketeer]

used different versions of kafka and zookeeper.

You're mixing up KAFKA_CFG_ADVERTISED_LISTENERS and KAFKA_ADVERTISED_LISTENERS, then

I think bitnami might have modified the names of those since I made this repo

[OneCricketeer]

localhost would not work, unless you're running this code outside of Docker

[vishnu-chalil]

Yes. Iam running the code outside the docker container.

[OneCricketeer]

Then that would explain why using Docker service names wouldn't work

If you'd like more details, read https://www.confluent.io/blog/kafka-listeners-explained/