Single Zookeeper / Single Kafka fails

[yngvark]

I am trying to fire up Kafka as described in the README, but I'm getting the error below.

• Os: Linux (kernel 5.4.0-66-generic)

$ docker-compose -f zk-single-kafka-single.yml up
Creating gr-z_zoo1_1 ... done
Creating gr-z_kafka1_1 ... done
Attaching to gr-z_zoo1_1, gr-z_kafka1_1
zoo1_1    | ZooKeeper JMX enabled by default
kafka1_1  | ===> User
zoo1_1    | Using config: /conf/zoo.cfg
zoo1_1    | 2021-03-08 18:37:16,207 [myid:] - INFO  [main:QuorumPeerConfig@124] - Reading configuration from: /conf/zoo.cfg
kafka1_1  | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
zoo1_1    | 2021-03-08 18:37:16,219 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: zoo1 to address: zoo1/172.18.0.2
zoo1_1    | 2021-03-08 18:37:16,219 [myid:] - ERROR [main:QuorumPeerConfig@301] - Invalid configuration, only one server specified (ignoring)
zoo1_1    | 2021-03-08 18:37:16,220 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
zoo1_1    | 2021-03-08 18:37:16,220 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
zoo1_1    | 2021-03-08 18:37:16,220 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
zoo1_1    | 2021-03-08 18:37:16,220 [myid:] - WARN  [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running  in standalone mode
kafka1_1  | ===> Configuring ...
zoo1_1    | 2021-03-08 18:37:16,229 [myid:] - INFO  [main:QuorumPeerConfig@124] - Reading configuration from: /conf/zoo.cfg
zoo1_1    | 2021-03-08 18:37:16,229 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: zoo1 to address: zoo1/172.18.0.2
zoo1_1    | 2021-03-08 18:37:16,229 [myid:] - ERROR [main:QuorumPeerConfig@301] - Invalid configuration, only one server specified (ignoring)
zoo1_1    | 2021-03-08 18:37:16,229 [myid:] - INFO  [main:ZooKeeperServerMain@96] - Starting server
zoo1_1    | 2021-03-08 18:37:16,233 [myid:] - INFO  [main:Environment@100] - Server environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
zoo1_1    | 2021-03-08 18:37:16,233 [myid:] - INFO  [main:Environment@100] - Server environment:host.name=zoo1
zoo1_1    | 2021-03-08 18:37:16,233 [myid:] - INFO  [main:Environment@100] - Server environment:java.version=1.8.0_121
zoo1_1    | 2021-03-08 18:37:16,233 [myid:] - INFO  [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
zoo1_1    | 2021-03-08 18:37:16,233 [myid:] - INFO  [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-1.8-openjdk/jre
zoo1_1    | 2021-03-08 18:37:16,234 [myid:] - INFO  [main:Environment@100] - Server environment:java.class.path=/zookeeper-3.4.9/bin/../build/classes:/zookeeper-3.4.9/bin/../build/lib/*.jar:/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/conf:
zoo1_1    | 2021-03-08 18:37:16,234 [myid:] - INFO  [main:Environment@100] - Server environment:java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
zoo1_1    | 2021-03-08 18:37:16,234 [myid:] - INFO  [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
zoo1_1    | 2021-03-08 18:37:16,234 [myid:] - INFO  [main:Environment@100] - Server environment:java.compiler=<NA>
zoo1_1    | 2021-03-08 18:37:16,234 [myid:] - INFO  [main:Environment@100] - Server environment:os.name=Linux
zoo1_1    | 2021-03-08 18:37:16,235 [myid:] - INFO  [main:Environment@100] - Server environment:os.arch=amd64
zoo1_1    | 2021-03-08 18:37:16,235 [myid:] - INFO  [main:Environment@100] - Server environment:os.version=5.4.0-66-generic
zoo1_1    | 2021-03-08 18:37:16,235 [myid:] - INFO  [main:Environment@100] - Server environment:user.name=zookeeper
zoo1_1    | 2021-03-08 18:37:16,235 [myid:] - INFO  [main:Environment@100] - Server environment:user.home=/home/zookeeper
zoo1_1    | 2021-03-08 18:37:16,235 [myid:] - INFO  [main:Environment@100] - Server environment:user.dir=/zookeeper-3.4.9
zoo1_1    | 2021-03-08 18:37:16,240 [myid:] - INFO  [main:ZooKeeperServer@815] - tickTime set to 2000
zoo1_1    | 2021-03-08 18:37:16,240 [myid:] - INFO  [main:ZooKeeperServer@824] - minSessionTimeout set to -1
zoo1_1    | 2021-03-08 18:37:16,240 [myid:] - INFO  [main:ZooKeeperServer@833] - maxSessionTimeout set to -1
zoo1_1    | 2021-03-08 18:37:16,247 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
kafka1_1  | ===> Running preflight checks ... 
kafka1_1  | ===> Check if /var/lib/kafka/data is writable ...
kafka1_1  | Command [/usr/local/bin/dub path /var/lib/kafka/data writable] FAILED !
gr-z_kafka1_1 exited with code 1

In case it helps:

$ uname -r
5.4.0-66-generic

$ docker-compose --version
docker-compose version 1.26.2, build eefe0d31

$ cd zk-single-kafka-single
$ tree
.
├── kafka1
│   └── data
└── zoo1
    ├── data
    │   ├── myid
    │   └── version-2
    └── datalog
        └── version-2

7 directories, 1 file

$ ll
drwxr-xr-x - root  8 mars  19:37  zoo1
drwxr-xr-x - root  8 mars  19:37  kafka1

[simplesteph]

hi!

1) Which version of Linux are you using? Is there a docker image / Amazon AMI I can use to reproduce this?

2) Are you using the latest version (master) ? Do you mind seeing if you see the same errors on this version? https://github.com/conduktor/kafka-stack-docker-compose/commit/f9aa1a641179e4ab412711bc7f86b4b7bfc5b0c1

Thanks Stephane

[yngvark]

1.

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"

I don't know how to reproduce this using docker. Use the ubuntu:20.04 image I guess? It would require you to use a docker-in-docker setup.

2.

This was master, yes.

Using f9aa1a6 works :+1: As in, both containers are up and running.

[simplesteph]

I've made a bug report here: https://github.com/confluentinc/kafka-images/issues/77

In the meantime I'm going to revert some commits here

[yngvark]

OK, thanks for quick follow up!

[simplesteph]

fixed by #93