search

mesos / kafka

Apache Kafka on Apache Mesos
Apache License 2.0
414 stars 140 forks source link

Master is not able to contact the scheduler brought up on docker #142

Closed rsingh2411 closed 9 years ago

rsingh2411 commented 9 years ago

Hi All,

I am trying to bring up scheduler using docker on one of the slaves. It seems that master is trying to send the acknowledgement to framework subscription to the scheduler framework.Interesting thing here is the ip which master is trying to contact is the docker containers ip.which is not accessible to master. Topology is on openstack with 1 master and two slaves and each machine has an public and private ip.

is there any configuration missing. Please advice. I haven't used marathon yet i am using following command to run the docker container

sudo docker run -t -p 5900:5900 --add-host=master:192.168.1.115 root/kafka-mesos ./kafka-mesos.sh scheduler --master=master:5050 --zk=192.168.1.115:2181 --api=http://192.168.1.118:5900 --storage=zk:/kafka-mesos --log=/var/log/mesos.log --debug=true

Received SUBSCRIBE call for framework 'kafka' at scheduler-b3b4006a-6389-456e-877b-107556d41c2d@172.17.0.8:36680 I1021 00:16:30.042913 28952 master.cpp:2164] Subscribing framework kafka with checkpointing enabled and capabilities [ ] I1021 00:16:30.042958 28952 master.cpp:2174] Framework 20151020-234515-1929488576-5050-28929-0002 (kafka) at scheduler-b3b4006a-6389-456e-877b-107556d41c2d@172.17.0.8:36680 already subscribed, resending acknowledgement W1021 00:16:30.043006 28952 master.hpp:1409] Master attempted to send message to disconnected framework 20151020-234515-1929488576-5050-28929-0002 (kafka) at scheduler-b3b4006a-6389-456e-877b-107556d41c2d@172.17.0.8:36680 I1021 00:16:35.762477 28951 http.cpp:321] HTTP GET for /master/state.json from 10.154.174.155:50091 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36' I1021 00:16:46.809695 28949 http.cpp:321] HTTP GET for /master/state.json from 10.154.174.155:50091 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36' I1021 00:16:57.808300 28953 http.cpp:321] HTTP GET for /master/state.json from 10.154.174.155:50091 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36' E1021 00:17:08.712221 28956 socket.hpp:174] Shutdown failed on fd=25: Transport endpoint is not connected [107]

rsingh2411 commented 9 years ago

Hi all,

sudo docker run -t -p 5900:5900 --net=host root/kafka-mesos ./kafka-mesos.sh scheduler --master=192.168.1.115:5050 --zk=192.168.1.115:2181 --api=http://192.168.1.118:5900 --storage=zk:/kafka-mesos --log=/var/log/mesos.log --debug=true

The issue got resolved by using --net=host why do i need to add --net=host, as it should work normally also. Can you please comment if anything is wrong.

SEJeff commented 9 years ago

You should run the framework via something like marathon or apache aurora, which defaults to running with --net=host. Running any production application via docker with the bridged networking mode is kind of discouraged. That is why mesos defaults to running apps via host networking mode.

rsingh2411 commented 9 years ago

Yeah it worked with running marathon as mentioned by you, there is an option in GUI to select network as host. Can we have multiple scheduler cluster running on one setup.

joestein commented 9 years ago

Yes you can have multiple schedulers running.