search

confluentinc / cp-docker-images

[DEPRECATED] Docker images for Confluent Platform.
Apache License 2.0
1.14k stars 704 forks source link

Unable to connect to local Kafka cluster from ControlCenter docker container #609

Closed DevonPeroutky closed 6 years ago

DevonPeroutky commented 6 years ago

I have kafka and zookeeper running on the same machine running as standalone processes. I am attempting to run the control center, on the same machine, using the following docker-compose file:

---
version: '2'
services:
  control-center:
    image: confluentinc/cp-enterprise-control-center:5.0.0
    hostname: control-center
    container_name: control-center
    ports:
      - "9021:9021"
    environment:
      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'http://localhost:9092'
      CONTROL_CENTER_ZOOKEEPER_CONNECT: 'http://localhost:2181'
      CONTROL_CENTER_REPLICATION_FACTOR: 1
      CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
      CONFLUENT_METRICS_TOPIC_REPLICATION: 1
      PORT: 9021

volumes:
    mi2: {}

I am able to connect to the cluster using console producer/consumers, but the ControlCenter has the following error:

...
control-center    | cub kafka-ready "${CONTROL_CENTER_REPLICATION_FACTOR}" \
control-center    |   "${CONTROL_CENTER_CUB_KAFKA_TIMEOUT:-40}" \
control-center    |   -b "${CONTROL_CENTER_BOOTSTRAP_SERVERS}" \
control-center    |   --config "${CONTROL_CENTER_CONFIG_DIR}/admin.properties"
control-center    | + cub kafka-ready 1 40 -b http://localhost:9092 --config /etc/confluent-control-center/admin.properties
control-center    | [main] INFO org.apache.kafka.clients.admin.AdminClientConfig - AdminClientConfig values: 
control-center    |     bootstrap.servers = [http://localhost:9092]
control-center    |     client.id = 
control-center    |     connections.max.idle.ms = 300000
control-center    |     metadata.max.age.ms = 300000
control-center    |     metric.reporters = []
control-center    |     metrics.num.samples = 2
control-center    |     metrics.recording.level = INFO
control-center    |     metrics.sample.window.ms = 30000
control-center    |     receive.buffer.bytes = 65536
control-center    |     reconnect.backoff.max.ms = 1000
control-center    |     reconnect.backoff.ms = 50
control-center    |     request.timeout.ms = 120000
control-center    |     retries = 5
control-center    |     retry.backoff.ms = 100
control-center    |     sasl.client.callback.handler.class = null
control-center    |     sasl.jaas.config = null
control-center    |     sasl.kerberos.kinit.cmd = /usr/bin/kinit
control-center    |     sasl.kerberos.min.time.before.relogin = 60000
control-center    |     sasl.kerberos.service.name = null
control-center    |     sasl.kerberos.ticket.renew.jitter = 0.05
control-center    |     sasl.kerberos.ticket.renew.window.factor = 0.8
control-center    |     sasl.login.callback.handler.class = null
control-center    |     sasl.login.class = null
control-center    |     sasl.login.refresh.buffer.seconds = 300
control-center    |     sasl.login.refresh.min.period.seconds = 60
control-center    |     sasl.login.refresh.window.factor = 0.8
control-center    |     sasl.login.refresh.window.jitter = 0.05
control-center    |     sasl.mechanism = GSSAPI
control-center    |     security.protocol = PLAINTEXT
control-center    |     send.buffer.bytes = 131072
control-center    |     ssl.cipher.suites = null
control-center    |     ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
control-center    |     ssl.endpoint.identification.algorithm = https
control-center    |     ssl.key.password = null
control-center    |     ssl.keymanager.algorithm = SunX509
control-center    |     ssl.keystore.location = null
control-center    |     ssl.keystore.password = null
control-center    |     ssl.keystore.type = JKS
control-center    |     ssl.protocol = TLS
control-center    |     ssl.provider = null
control-center    |     ssl.secure.random.implementation = null
control-center    |     ssl.trustmanager.algorithm = PKIX
control-center    |     ssl.truststore.location = null
control-center    |     ssl.truststore.password = null
control-center    |     ssl.truststore.type = JKS
control-center    | 
control-center    | [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version : 2.0.0-cpNone
control-center    | [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId : ca8d91be74ec83ed
control-center    | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
control-center    | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
control-center    | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
control-center    | [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
control-center    | java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
control-center    |     at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
control-center    |     at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
control-center    |     at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
control-center    |     at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262)
control-center    |     at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:149)
control-center    |     at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
control-center    | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
control-center    | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
control-center    | [main] INFO io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Trying to query Kafka for metadata again ...
control-center    | [main] ERROR io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Brokers found [].

Any idea what could be going on? Do the brokers need extra configuration in the server.properties?

jcustenborder commented 6 years ago

hi @DevonPeroutky

Here is your problem. 

      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'http://localhost:9092'
      CONTROL_CENTER_ZOOKEEPER_CONNECT: 'http://localhost:2181'

should be 

      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'localhost:9092'
      CONTROL_CENTER_ZOOKEEPER_CONNECT: 'localhost:2181'

Kafka and Zookeeper are their own TCP protocols and are not http based. Also make sure those are correct. You're basically saying that the control center container should connect to it's container. Zookeeper and Kafka are not running there. Look at cp-demo for a good example.