search

mesosphere-backup / cassandra-mesos-deprecated

[DEPRECATED] This project is deprecated. It will be archived on December 1, 2017.
https://github.com/mesosphere/mesosphere/dcos-commons/frameworks/cassandra.
Apache License 2.0
184 stars 68 forks source link

Launch on Mesos/Marathon created with docker compose #157

Closed kiambogo closed 8 years ago

kiambogo commented 9 years ago

I have Mesos/Zookeeper/Marathon/slave nodes created in docker containers, and am trying to add cassandra-mesos with Marathon. The application is stuck in Deploying in Marathon. In Mesos, I can see the task while it's staging and can see through the sandbox that it downloads the necessary files. However, the task gets killed shortly after with no indication of what went wrong in stderr/stdout. Tailing the logs of the docker slave node, I can see the following:

I1109 19:25:57.967133     8 slave.cpp:1270] Got assigned task cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961 for framework f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000
I1109 19:25:57.967419     8 slave.cpp:1386] Launching task cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961 for framework f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000
I1109 19:25:58.021013     8 slave.cpp:4852] Launching executor cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961 of framework f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000 with resources cpus(*):0.1; mem(*):32 in work directory '/tmp/mesos/slaves/f5aed497-2f9e-4b5f-862e-e6a4ad94e306-S0/frameworks/f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000/executors/cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961/runs/95424c8c-77f9-4ff8-becb-4e4cf1e1cfaa'
I1109 19:25:58.022163    10 docker.cpp:734] No container info found, skipping launch
I1109 19:25:58.023249    10 containerizer.cpp:640] Starting container '95424c8c-77f9-4ff8-becb-4e4cf1e1cfaa' for executor 'cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961' of framework 'f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000'
I1109 19:25:58.028839     8 slave.cpp:1604] Queuing task 'cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961' for executor cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961 of framework 'f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000
I1109 19:25:58.030902    10 linux_launcher.cpp:352] Cloning child process with flags =
I1109 19:25:58.036380    10 containerizer.cpp:873] Checkpointing executor's forked pid 861 to '/tmp/mesos/meta/slaves/f5aed497-2f9e-4b5f-862e-e6a4ad94e306-S0/frameworks/f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000/executors/cassandra_dev-test.b40c7218-8717-11e5-98a7-760869961961/runs/95424c8c-77f9-4ff8-becb-4e4cf1e1cfaa/pids/forked.pid'
E1109 19:26:08.176664    11 slave.cpp:3342] Container 'eb670e50-c40a-43ee-b577-9a76df5fab96' for executor 'cassandra_dev-test.ab14e9d7-8717-11e5-98a7-760869961961' of framework 'f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000' failed to start: Container destroyed during launch
E1109 19:26:08.182646    11 slave.cpp:3424] Termination of executor 'cassandra_dev-test.ab14e9d7-8717-11e5-98a7-760869961961' of framework 'f5aed497-2f9e-4b5f-862e-e6a4ad94e306-0000' failed:     Unknown container: eb670e50-c40a-43ee-b577-9a76df5fab96`

Can cassandra-mesos be launched in this manner (dockerized)? Below if the docker compose template I'm using (on OS X).

zk:
  image: bobrik/zookeeper
  net: host
  ports:
   - 2181:2181
  environment:
    ZK_CONFIG: tickTime=2000,initLimit=10,syncLimit=5,maxClientCnxns=128,forceSync=no,clientPort=2181
    ZK_ID: 1

master:
  image: mesosphere/mesos-master:0.25.0-0.2.70.ubuntu1404
  net: host
  environment:
    MESOS_ZK: zk://192.168.99.100:2181/mesos
    MESOS_QUORUM: 1
    MESOS_CLUSTER: docker-compose
    MESOS_WORK_DIR: /var/lib/mesos

slave:
  image: mesosphere/mesos-slave:0.25.0-0.2.70.ubuntu1404
  net: host
  privileged: true
  environment:
    MESOS_MASTER: zk://192.168.99.100:2181/mesos
    MESOS_CONTAINERIZERS: docker,mesos
    MESOS_PORT: 5051
    MESOS_RESOURCES: ports(*):[31000-32000,7000-7001,7199-7199,9042-9042,9160-9160];mem(*):5000;cpus(*):4
  volumes:
    - /sys/fs/cgroup:/sys/fs/cgroup
    - /usr/local/bin/docker:/usr/bin/docker
    - /var/run/docker.sock:/var/run/docker.sock

marathon:
  image: mesosphere/marathon:v0.11.0
  net: host
  environment:
    MARATHON_MASTER: zk://192.168.99.100:2181/mesos
    MARATHON_DECLINE_OFFER_DURATION: 3600000
alfredcs commented 8 years ago

We ran into similar situation while the same Cassandra image worked just fine when deploying with docker directly. The error seemed to be caused by "cassnadra -f" exited with "Killed" in docker-entrypoint.sh under marathon. No ROC or work-around has been identified yet.

triclambert commented 8 years ago

This repository is deprecated. Please use the latest supported framework here: https://github.com/mesosphere/dcos-cassandra-service, and provide feedback for us here: https://dcosjira.atlassian.net/browse/CASSANDRA.