search

confluentinc / cp-all-in-one

docker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud, Apache Kafka Confluent Platform
959 stars 684 forks source link

Exit code 1 for multiple containers, missing Dockerfile #53

Open marcin2x4 opened 3 years ago

marcin2x4 commented 3 years ago

Below services end with Exit 1 code:

I tried adding memory limit but no luck, should these be placed in specific line of docker-compose file?

deploy:
      resources:
        limits:
          cpus: xxx
          memory: xxx
        reservations:
          cpus: xxx
          memory: xxx

Also I would suggest to add to each docker-compose a Dockerfile as well.

ybyzek commented 3 years ago

@marcin2x4 what are the error messages? docker-compose logs <name of container>

marcin2x4 commented 3 years ago

@ybyzek - very long ;) Shall I zip them?

ybyzek commented 3 years ago

@marcin2x4 can you view the log and pull out the ERROR messages?

marcin2x4 commented 3 years ago

@ybyzek - as I ran docker-compose logs schema-registry I get below errors:

schema-registry | org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1606854333491) timed out at 1606854333492 after 1 attempt(s)

schema-registry | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.

schema-registry | [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.

marcin2x4 commented 3 years ago

confluent_exit1_error.txt @ybyzek - file attached for schema-registry

ybyzek commented 3 years ago

The last part of the Schema Registry log suggests an issue with the broker? What do broker logs say?

schema-registry    | [kafka-admin-client-thread | adminclient-1] INFO org.apache.kafka.clients.admin.internals.AdminMetadataManager - [AdminClient clientId=adminclient-1] Metadata update failed
schema-registry    | org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1607371198629) timed out at 1607371198630 after 1 attempt(s)
schema-registry    | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
schema-registry    | [main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
schema-registry    | java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1607371208630) timed out at 1607371208631 after 1 attempt(s)
schema-registry    |   at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
schema-registry    |   at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
schema-registry    |   at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
schema-registry    |   at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
schema-registry    |   at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:149)
schema-registry    |   at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
schema-registry    | Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1607371208630) timed out at 1607371208631 after 1 attempt(s)
schema-registry    | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
schema-registry    | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/172.28.0.3:29092) could not be established. Broker may not be available.
schema-registry    | [main] INFO io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Trying to query Kafka for metadata again ...
schema-registry    | [main] ERROR io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Brokers found [].
marcin2x4 commented 3 years ago

Due to size I ziped the thing. broker_error_exit1.zip

ybyzek commented 3 years ago

Some errors from the logs:

^[[36mbroker             |^[[0m [2020-09-19 23:45:41,196] ERROR [Controller id=1] Error while electing or becoming controller on broker 1. Trigger controller movement immediately (kafka.controller.KafkaController)
^[[36mbroker             |^[[0m org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
^[[36mbroker             |^[[0m [2020-09-19 23:50:51,291] ERROR Error while creating ephemeral at /brokers/ids/1, node already exists and owner '72063151865331727' does not match current session '72057602203713540' (kafka.zk.KafkaZkClient$CheckedEphemeral)

Is there another instance of Kafka or ZooKeeper running locally? (either Docker or local install)

marcin2x4 commented 3 years ago

@ybyzek - no, only the one I mentioned. Here is dockerfile and docker-compose I use to run this environment. https://github.com/marcin2x4/confluent_kafka_docker_image

tooptoop4 commented 2 years ago

@marcin2x4 did u solve?