search

sknop / kafka-docker-composer

Python script to generate a docker-compose.yaml file based on templates and parameters
Apache License 2.0
63 stars 17 forks source link

KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE contains true, which is an invalid type, it should be a string, number, or a null #2

Open danjrichards opened 7 months ago

danjrichards commented 7 months ago

Hi Sven Nice tool - thank you.

I ran python3 kafka_docker_composer.py -b 3 -z 3 --racks 3 and it generated 3 errors on starting, each: services.kafka-?.environment.KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE contains true, which is an invalid type, it should be a string, number, or a null

I changed each instance to KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE="true" and this error went away, but I then get: Error opening zip file or JAR manifest missing : /tmp/jmx_prometheus_javaagent-0.20.0.jar zookeeper-2 | agent library failed to init: instrument

danjrichards commented 7 months ago

instrumenting error was my local issue - for some reason jmx_prometheus_javaagent-0.20.0.jar was an empty directory! 🤷🏼

sknop commented 6 months ago

I am not quite sure how you got these errors, I cannot reproduce any of this.

Which OS are you running, and which docker engine do you use?

DominiqueLuebben commented 2 weeks ago

Hello,

I am getting the same errors like danjrichards. I am trying to setup "python kafka_docker_composer.py --brokers 1 --controllers 1 --p". I get errors because of "KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE: False" and "KAFKA_CONFLUENT_REPORTERS_TELEMETRY_AUTO_ENABLE: False". Additionally there is no prometheus.yml in the volumes folder

sknop commented 2 weeks ago

Hi Dominique,

Can you try three brokers and three controllers first to see if that works for you? There are some extra services enabled in the Confluent server that are set to a replication factor of 3 by default, it is possible I missed a few.

I will investigate.

sknop commented 2 weeks ago

Also, it is -p, not --p?

Kafka docker-compose Generator

options: -h, --help show this help message and exit -r RELEASE, --release RELEASE Docker images release [7.7.0] --with-tc Build and use a local image with tc enabled --shared-mode Enable shared mode for controllers -b BROKERS, --brokers BROKERS Number of Brokers [1] -z ZOOKEEPERS, --zookeepers ZOOKEEPERS Number of ZooKeepers [0] - mutually exclusive with controllers -c CONTROLLERS, --controllers CONTROLLERS Number of Kafka controller instances [0] - mutually exclusive with ZooKeepers -s SCHEMA_REGISTRIES, --schema-registries SCHEMA_REGISTRIES Number of Schema Registry instances [0] -C CONNECT_INSTANCES, --connect-instances CONNECT_INSTANCES Number of Kafka Connect instances [0] -k KSQLDB_INSTANCES, --ksqldb-instances KSQLDB_INSTANCES Number of ksqlDB instances [0] --control-center Include Confluent Control Center [False] --uuid UUID UUID of the cluster [Nk018hRAQFytWskYqtQduw] -p, --prometheus Include Prometheus [False] --kafka-container KAFKA_CONTAINER Container used for Kafka, default [cp-server] --racks RACKS Number of racks among which the brokers will be distributed evenly [1] --zookeeper-groups ZOOKEEPER_GROUPS Number of zookeeper groups in a hierarchy [1] --docker-compose-file DOCKER_COMPOSE_FILE Output file for docker-compose, default [docker-compose.yml] --config CONFIG Properties config file, values will be overridden by command line arguments