confluentinc / confluent-cli

Confluent Platform CLI
Other
60 stars 38 forks source link

Confluent CLI doesn't generate ZooKeeper config having more than 1 instance. #46

Open SoyeeDst opened 7 years ago

SoyeeDst commented 7 years ago

If you install CP from DEB/RPM, you have /etc/kafka/zookeeper.properties which says that the serve.x is the server information for each node. So if you do “confluent start” to start zookeeper, you may expect the current node will join the cluster in the section of server.x group. But actually it wasn't.

One magic file /tmp/confluent.zJ3sbPTi/zookeeper/zookeeper.properties was generated in my AWS run, however that makes sense with the explanation of Confluent CLI doesn't use the standard Confluent Config with no warning!. My running log still shows that the file to specify Zookeeper myid must be required as well though, here comes the complete stack trace.

[2017-09-25 20:45:54,120] INFO Defaulting to majority quorums (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [2017-09-25 20:45:54,121] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain) org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /tmp/confluent.zJ3sbPTi/zookeeper/zookeeper.properties at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:154) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) Caused by: java.lang.IllegalArgumentException: /tmp/confluent.zJ3sbPTi/zookeeper/data/myid file is missing at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:406) at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:150)

mmuehlbeyer commented 7 years ago

same issue here starting with zookeeper-server-start works as expected

OSEDELTA commented 7 years ago

The same for me.

So I would like to know how should we start/configure confluent CLI to use the configuration that is written under /etc/kafka and /etc/schema-registry (which are the standard places created when you install confluent via apt-get/deb)

jarnix commented 6 years ago

It looks like that it's the same issue with the tarball, the tmp directory does not contain the myid and zookeeper doesn't start. In my test server the myid file is located in the right folder (starting with zookeeper-server-start is fine)

SoyeeDst commented 6 years ago

@jarnix yes, zookeeper-server-start works fine in my environment as well

danielec7 commented 6 years ago

I have the same issue.

dimon222 commented 6 years ago

Pretty much same issue here. I'm currently rewriting myid file manually after folder structure in /tmp is created (after failed launch) to workaround this problem.

windman83 commented 6 years ago

same issue!!

windman83 commented 6 years ago

workaround for me was adding cp /var/lib/zookeeper/myid $confluent_current/zookeeper/data/ after config_service "zookeeper" "kafka" "zookeeper" "dataDir" in /bin/confluent