eucuepo / vagrant-kafka

Vagrant config to setup a partitioned Apache Kafka installation with clustered Apache Zookeeper
103 stars 89 forks source link

Getting error when create or listing topics #6

Closed caddac closed 7 years ago

caddac commented 7 years ago

Running the create_topics.sh or list-topics.sh script give me the following error over and over until I ctrl + c to quit:

[vagrant@broker1 ~]$ /vagrant/scripts/create_topic.sh myTopic
[2017-01-15 18:49:40,997] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.
ClientCnxn)
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
imarios commented 7 years ago

Can you verify that ZooKeeper nodes are up?

vagrant ssh zookeeper1 -c jps
# then
vagrant ssh zookeeper2 -c jps
# then
vagrant ssh zookeeper3 -c jps

You should see a QuorumPeerMain in each zk node.

caddac commented 7 years ago

Output of commands:

D:\Repositories\vagrant-kafka [master ≡ +0 ~2 -0 !]> vagrant ssh zookeeper1 -c jps
3637 Jps
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~2 -0 !]> vagrant ssh zookeeper2 -c jps
3635 Jps
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~2 -0 !]> vagrant ssh zookeeper3 -c jps
3642 Jps
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~2 -0 !]>

Not sure how to see a QuorumPeerMain?

imarios commented 7 years ago

For some reason Zk has not started. Start it using:

vagrant ssh zookeeper1 -c "/vagrant/scripts/zookeeper.sh 1"
vagrant ssh zookeeper2 -c "/vagrant/scripts/zookeeper.sh 2"
vagrant ssh zookeeper3 -c "/vagrant/scripts/zookeeper.sh 3"

After this,

vagrant ssh zookeeper3 -c jps

Should show you a QuorumPeerMain

caddac commented 7 years ago

These commands appeared to work after running the sed command on each script (#5). However no change when i ran the final command vagrant ssh zookeeper3 -c jps

D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper1 -c "sed -i -e 's/\r$//' /vagrant/scripts/zookeeper.sh"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper2 -c "sed -i -e 's/\r$//' /vagrant/scripts/zookeeper.sh"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper3 -c "sed -i -e 's/\r$//' /vagrant/scripts/zookeeper.sh"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper1 -c "/vagrant/scripts/zookeeper.sh 1"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper2 -c "/vagrant/scripts/zookeeper.sh 2"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper3 -c "/vagrant/scripts/zookeeper.sh 3"
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper3 -c jps
3764 Jps
Connection to 127.0.0.1 closed.

However, I connected to the zookeeper3 via ssh and ran the zookeeper.sh file and I got QuorumPeerMain, but it appears to have died immediately.

D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper1
Last login: Sun Jan 15 21:00:46 2017 from 10.0.2.2
[vagrant@zookeeper1 ~]$ cd /vagrant/scripts/
[vagrant@zookeeper1 scripts]$ ./zookeeper.sh
[vagrant@zookeeper1 scripts]$ jps
3941 Jps
3917 QuorumPeerMain
[vagrant@zookeeper1 scripts]$ exit
logout
Connection to 127.0.0.1 closed.
D:\Repositories\vagrant-kafka [master ≡ +0 ~3 -0 !]> vagrant ssh zookeeper3 -c jps
3800 Jps
Connection to 127.0.0.1 closed.

Thanks for your help!

imarios commented 7 years ago

Try when you start zk to give an id

[vagrant@zookeeper1 scripts]$ ./zookeeper.sh 1
imarios commented 7 years ago

I think you need to run your cleanup (windows line feed) script in all the files. All the scripts and all the configurations. Something must be going wrong when zk tries to read its configuration file.

caddac commented 7 years ago

I created a pull request that solves both my issues by forcing the right line endings. I verified it works by forking and adding the .gitattributes file. I was able to successfully create topics without running the sed command on any scripts.