search

mesos / docker-compose-executor

DEPRECATED: Find the new docker compose executor here https://github.com/paypal/dce-go
Apache License 2.0
56 stars 16 forks source link

Integration with Running Mesos/Marathon #9

Closed akamalov closed 8 years ago

akamalov commented 8 years ago

Environment

RHEL7.2
Marathon: 0.14.0
Mesos: 0.26.0
Docker-Engine: 1.10.3

Running a full-blown Mesos/Marathon ecosystem. Build docker-compose-executor without any problem. Copied the docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar to Mesos-Slave, as well as docker-compose-executor.sh file.

Here are the docker-compose-executor.sh file:

File: docker-compose-executor.sh - on Mesos-Slave node

COMPOSE_JAR_NAME=/opt/docker-compose-executor/docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar
COMPOSE_CLASS_NAME=com.paypal.mesos.executor.App
java -cp ${COMPOSE_JAR_NAME} ${COMPOSE_CLASS_NAME}

I am attempting to deploy Docker-Notary which comprises of three containers:

Here is my docker-compose.yml file:

File: docker-compose.yml

server:
  build: .
  dockerfile: server.Dockerfile
  links:
    - mysql
    - signer
    - signer:notarysigner
  environment:
    - SERVICE_NAME=notary_server
  ports:
    - "8080:18080"
    - "4443:14443"
  entrypoint: /bin/bash
  command: -c "./migrations/migrate.sh && notary-server -config=fixtures/server-config.json"
signer:
  build: .
  dockerfile: signer.Dockerfile
  links:
    - mysql
  environment:
    - SERVICE_NAME=notary_signer
  ports:
    -"4444:14444"
  entrypoint: /bin/bash
  command: -c "./migrations/migrate.sh && notary-signer -config=fixtures/signer-config.json"
mysql:
  volumes:
    - ./notarymysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
    - notary_data:/var/lib/mysql
  image: mariadb:10.1.10
  ports:
    - "3306:13306"
  environment:
    - TERM=dumb
    - MYSQL_ALLOW_EMPTY_PASSWORD="true"
  command: mysqld --innodb_file_per_table

As shown above, this file will call on two additional files:

File: server.Dockerfile

FROM golang:1.6.0
MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \
    libltdl-dev \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

EXPOSE 4443

# Install DB migration tool
RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary

# Copy the local repo to the expected go path
COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

# Install notary-server
RUN go install \
    -tags pkcs11 \
    -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=`git rev-parse --short HEAD` -X ${NOTARYPKG}/version.NotaryVersion=`cat NOTARY_VERSION`" \
    ${NOTARYPKG}/cmd/notary-server

ENTRYPOINT [ "notary-server" ]
CMD [ "-config=fixtures/server-config-local.json" ]

File: signer.Dockerfile

FROM golang:1.6.0
MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \
    libltdl-dev \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

EXPOSE 4444

# Install DB migration tool
RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary
ENV NOTARY_SIGNER_DEFAULT_ALIAS="timestamp_1"
ENV NOTARY_SIGNER_TIMESTAMP_1="testpassword"

# Copy the local repo to the expected go path
COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

# Install notary-signer
RUN go install \
    -tags pkcs11 \
    -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=`git rev-parse --short HEAD` -X ${NOTARYPKG}/version.NotaryVersion=`cat NOTARY_VERSION`" \
    ${NOTARYPKG}/cmd/notary-signer

ENTRYPOINT [ "notary-signer" ]
CMD [ "-config=fixtures/signer-config-local.json" ]

Now, I am trying to deploy these containers using the following JSON file:

File: notary-composer.json

{
    "id": "dockernotary",
    "cmd": " ",
    "instances": 1,
    "constraints": [["hostname", "CLUSTER", "node264.local.net"]],
    "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh",
    "labels": {
        "fileName": "/opt/docker-compose-executor/etc/docker-compose.yml"
    }
}

I tried to submit it to /apps and /groups. Attempt to submit to /apps resulted in a deployment going into a Waiting state.

Any clues ?

Thanks!!

Alex

mbdas commented 8 years ago

Do you see the task getting launched in mesos master console at all?

Sent from my iPhone

On Mar 18, 2016, at 8:19 AM, akamalov notifications@github.com wrote:

Environment

RHEL7.2 Marathon: 0.14.0 Mesos: 0.26.0 Docker-Engine: 1.10.3 Running a full-blown Mesos/Marathon ecosystem. Build docker-compose-executor without any problem. Copied the docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar to Mesos-Slave, as well as docker-compose-executor.sh file.

Here are the docker-compose-executor.sh file:

File: docker-compose-executor.sh - on Mesos-Slave node

COMPOSE_JAR_NAME=/opt/docker-compose-executor/docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar COMPOSE_CLASS_NAME=com.paypal.mesos.executor.App java -cp ${COMPOSE_JAR_NAME} ${COMPOSE_CLASS_NAME} I am attempting to deploy Docker-Notary which comprises of three containers:

Notary MySQL Notary Server Notary Signer Here is my docker-compose.yml file:

File: docker-compose.yml

server: build: . dockerfile: server.Dockerfile links:

  • mysql
  • signer
  • signer:notarysigner environment:
  • SERVICE_NAME=notary_server ports:
  • "8080:18080"
  • "4443:14443" entrypoint: /bin/bash command: -c "./migrations/migrate.sh && notary-server -config=fixtures/server-config.json" signer: build: . dockerfile: signer.Dockerfile links:
  • mysql environment:
  • SERVICE_NAME=notary_signer ports: -"4444:14444" entrypoint: /bin/bash command: -c "./migrations/migrate.sh && notary-signer -config=fixtures/signer-config.json" mysql: volumes:
  • ./notarymysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
  • notary_data:/var/lib/mysql image: mariadb:10.1.10 ports:
  • "3306:13306" environment:
  • TERM=dumb
  • MYSQL_ALLOW_EMPTY_PASSWORD="true" command: mysqld --innodb_file_per_table

As shown above, this file will call on two additional files:

server.Dockerfile signer.Dockerfile File: server.Dockerfile

FROM golang:1.6.0 MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \ libltdl-dev \ --no-install-recommends \ && rm -rf /var/lib/apt/lists/*

EXPOSE 4443

Install DB migration tool

RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary

Copy the local repo to the expected go path

COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

Install notary-server

RUN go install \ -tags pkcs11 \ -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=git rev-parse --short HEAD -X ${NOTARYPKG}/version.NotaryVersion=cat NOTARY_VERSION" \ ${NOTARYPKG}/cmd/notary-server

ENTRYPOINT [ "notary-server" ] CMD [ "-config=fixtures/server-config-local.json" ] File: signer.Dockerfile

FROM golang:1.6.0 MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \ libltdl-dev \ --no-install-recommends \ && rm -rf /var/lib/apt/lists/*

EXPOSE 4444

Install DB migration tool

RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary ENV NOTARY_SIGNER_DEFAULT_ALIAS="timestamp_1" ENV NOTARY_SIGNER_TIMESTAMP_1="testpassword"

Copy the local repo to the expected go path

COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

Install notary-signer

RUN go install \ -tags pkcs11 \ -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=git rev-parse --short HEAD -X ${NOTARYPKG}/version.NotaryVersion=cat NOTARY_VERSION" \ ${NOTARYPKG}/cmd/notary-signer

ENTRYPOINT [ "notary-signer" ] CMD [ "-config=fixtures/signer-config-local.json" ] Now, I am trying to deploy these containers using the following JSON file:

File: notary-composer.json

{ "id": "dockernotary", "cmd": " ", "instances": 1, "constraints": [["hostname", "CLUSTER", "node264.local.net"]], "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh", "labels": { "fileName": "/opt/docker-compose-executor/etc/docker-compose.yml" } } I tried to submit it to /apps and /groups. Attempt to submit to /apps resulted in a deployment going into a Waiting state.

Any clues ?

Thanks!!

Alex

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

mbdas commented 8 years ago

Also unless there is a reason for you to build the image in mesos slave itself, the compose file should just indicate the image to use and skip build/dockerfiles.

Sent from my iPhone

On Mar 18, 2016, at 8:19 AM, akamalov notifications@github.com wrote:

Environment

RHEL7.2 Marathon: 0.14.0 Mesos: 0.26.0 Docker-Engine: 1.10.3 Running a full-blown Mesos/Marathon ecosystem. Build docker-compose-executor without any problem. Copied the docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar to Mesos-Slave, as well as docker-compose-executor.sh file.

Here are the docker-compose-executor.sh file:

File: docker-compose-executor.sh - on Mesos-Slave node

COMPOSE_JAR_NAME=/opt/docker-compose-executor/docker-compose-executor-0.0.1-SNAPSHOT-jar-with-dependencies.jar COMPOSE_CLASS_NAME=com.paypal.mesos.executor.App java -cp ${COMPOSE_JAR_NAME} ${COMPOSE_CLASS_NAME} I am attempting to deploy Docker-Notary which comprises of three containers:

Notary MySQL Notary Server Notary Signer Here is my docker-compose.yml file:

File: docker-compose.yml

server: build: . dockerfile: server.Dockerfile links:

  • mysql
  • signer
  • signer:notarysigner environment:
  • SERVICE_NAME=notary_server ports:
  • "8080:18080"
  • "4443:14443" entrypoint: /bin/bash command: -c "./migrations/migrate.sh && notary-server -config=fixtures/server-config.json" signer: build: . dockerfile: signer.Dockerfile links:
  • mysql environment:
  • SERVICE_NAME=notary_signer ports: -"4444:14444" entrypoint: /bin/bash command: -c "./migrations/migrate.sh && notary-signer -config=fixtures/signer-config.json" mysql: volumes:
  • ./notarymysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
  • notary_data:/var/lib/mysql image: mariadb:10.1.10 ports:
  • "3306:13306" environment:
  • TERM=dumb
  • MYSQL_ALLOW_EMPTY_PASSWORD="true" command: mysqld --innodb_file_per_table

As shown above, this file will call on two additional files:

server.Dockerfile signer.Dockerfile File: server.Dockerfile

FROM golang:1.6.0 MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \ libltdl-dev \ --no-install-recommends \ && rm -rf /var/lib/apt/lists/*

EXPOSE 4443

Install DB migration tool

RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary

Copy the local repo to the expected go path

COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

Install notary-server

RUN go install \ -tags pkcs11 \ -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=git rev-parse --short HEAD -X ${NOTARYPKG}/version.NotaryVersion=cat NOTARY_VERSION" \ ${NOTARYPKG}/cmd/notary-server

ENTRYPOINT [ "notary-server" ] CMD [ "-config=fixtures/server-config-local.json" ] File: signer.Dockerfile

FROM golang:1.6.0 MAINTAINER David Lawrence "david.lawrence@docker.com"

RUN apt-get update && apt-get install -y \ libltdl-dev \ --no-install-recommends \ && rm -rf /var/lib/apt/lists/*

EXPOSE 4444

Install DB migration tool

RUN go get github.com/mattes/migrate

ENV NOTARYPKG github.com/docker/notary ENV NOTARY_SIGNER_DEFAULT_ALIAS="timestamp_1" ENV NOTARY_SIGNER_TIMESTAMP_1="testpassword"

Copy the local repo to the expected go path

COPY . /go/src/github.com/docker/notary

WORKDIR /go/src/${NOTARYPKG}

Install notary-signer

RUN go install \ -tags pkcs11 \ -ldflags "-w -X ${NOTARYPKG}/version.GitCommit=git rev-parse --short HEAD -X ${NOTARYPKG}/version.NotaryVersion=cat NOTARY_VERSION" \ ${NOTARYPKG}/cmd/notary-signer

ENTRYPOINT [ "notary-signer" ] CMD [ "-config=fixtures/signer-config-local.json" ] Now, I am trying to deploy these containers using the following JSON file:

File: notary-composer.json

{ "id": "dockernotary", "cmd": " ", "instances": 1, "constraints": [["hostname", "CLUSTER", "node264.local.net"]], "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh", "labels": { "fileName": "/opt/docker-compose-executor/etc/docker-compose.yml" } } I tried to submit it to /apps and /groups. Attempt to submit to /apps resulted in a deployment going into a Waiting state.

Any clues ?

Thanks!!

Alex

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

akamalov commented 8 years ago

Thanks for the answer. Replying the questions:

  1. Do you see the task getting launched in mesos master console at all?

A: Yes, task gets launched, but then enters Waiting... state

  1. ...just indicate the image to use and skip build/dockerfiles

A: Will try doing this next.

mbdas commented 8 years ago

You can probably look at mesos master console to see the task sandbox and logs under it to have clues. I looked at your above example and I didn't see you passing some sort of zip file to marathon to download the entire bundle of files that not only contains your docker compose file but the dockerfiles, etc needed for your docker build. If you change compose file to just point to an image, then you still need to pass the compose file directly or in a zip for it to be downloaded in task sandbox.

https://github.com/mesos/docker-compose-executor/blob/master/docs/getting-started.md (see the uris field)

If you are sending all the files, then looking at errors in logs in task sandbox may provide some clue.

akamalov commented 8 years ago

Thanks for the response. What I did was to pass on docker-compose file directly:

notary-compose.json

{
    "id": "dockernotary",
    "cmd": " ",
    "instances": 1,
    "constraints": [["hostname", "CLUSTER", "node264.local.net"]],
    "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh",
    "labels": {
        "fileName": "/opt/docker-compose-executor/etc/docker-compose.yml"
    }
}

As you can see above, I am calling directly docker-compose.yml file.

Now, this file (docker-compose.yml) references Dockerfiles:

server.Dockerfile signer.Dockerfile

...which are placed in the same directory as the docker-compose.yml.

Sad part is that when launch this app, Marathon just sits there displaying "Waiting" :(

If I submit images, instead of building the apps, for some reason notary-servers can't discover a database server (might be a linking issue, but that's another issue).

akamalov commented 8 years ago

Did the basic "hello world" docker-compose test:

test-compose.json

{
    "id": "test1",
    "cmd": "echo Hello World",
    "instances": 1,
    "constraints": [["hostname", "CLUSTER", "node264.local.net"]],
    "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh",
    "labels": {
        "fileName": "/opt/test/docker-compose.yml"
    }
}

Submitted to Marathon. The same behavior, Marathon goes into Waiting state.Looked up on Mesos-Slave for the errors, getting the following:

Mar 21 08:11:02 node264 mesos-slave[1524]: I0321 08:11:02.025629  1979 slave.cpp:3553] Executor 'marathon-test1.fa541788-ef5d-11e5-9c76-02429ce31955' of framework 7901cb4e-1e48-4a06-a85b-f340322ad02b-0000 exited with status 127
Mar 21 08:11:02 node264 mesos-slave[1524]: I0321 08:11:02.027550  1979 slave.cpp:2762] Handling status update TASK_FAILED (UUID: 7a61b392-e3a2-40fb-b722-8880cc10a703) for task test1.fa541788-ef5d-11e5-9c76-02429ce31955 of framework 7901cb4e-1e48-4a06-a85b-f340322ad02b-0000 from @0.0.0.0:0

Alex

mbdas commented 8 years ago

Are you setting a uri in marathon request?

Sent from my iPhone

On Mar 21, 2016, at 4:56 AM, akamalov notifications@github.com wrote:

Did the basic "hello world" docker-compose test:

test-compose.json

{ "id": "test1", "cmd": "echo Hello World", "instances": 1, "constraints": [["hostname", "CLUSTER", "node264.local.net"]], "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh", "labels": { "fileName": "/opt/test/docker-compose.yml" } } Submitted to Marathon. The same behavior, Marathon goes into Waiting state.

Alex

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

gtejasvarma commented 8 years ago

@akamalov can you please check if docker-compose-generated.yml is generated.

To run multiple pods run on the same machine and avoid port collision we treat port as a resource and replace host ports in the generated docker file. so try if this request works.

{
    "id": "test1",
    "cmd": "echo Hello World",
    "instances": 1,
     "ports":[0,0,0,0],
    "constraints": [["hostname", "CLUSTER", "node264.local.net"]],
    "executor":"/opt/docker-compose-executor/bin/docker-compose-executor.sh",
    "labels": {
        "fileName": "/opt/test/docker-compose.yml"
    }
}
mbdas commented 8 years ago

Unless there are further questions going to close the issue.