search

docker-archive / deploykit

A toolkit for creating and managing declarative, self-healing infrastructure.
Apache License 2.0
2.25k stars 262 forks source link

swarm-vagrant-manager.json writes to /etc/default/docker which does not work on systemd systems #325

Open dougbeauchene opened 7 years ago

dougbeauchene commented 7 years ago

I am trying to deploy Infrakit in a Ubuntu 16.04 vm running via VirtualBox on my Windows laptop.

I'm running into an issue that when I try to fire up the swarm-vagrant-workers using... src/github.com/docker/infrakit/build/infrakit group commit src/github.com/docker/infrakit/pkg/example/flavor/swarm/swarm-vagrant-workers.json

I believe the root issue is the docker daemon cannot be reached.

In swarm-vagrant-manager.json I see /etc/default/docker being updated with DOCKER_OPTS...

"Init": [
          "docker swarm init --advertise-addr 192.168.2.200",
          "echo 'DOCKER_OPTS=\"$DOCKER_OPTS -H tcp://0.0.0.0:4243\"' >> /etc/default/docker",
          "service docker restart"
]

But according to what I'm reading (https://github.com/docker/docker/issues/25357), /etc/default/docker is only used on systems running upstart and sysvinit, not on systems using systemd like Ubuntu 16.04 which I'm using.

So the setting of DOCKER_OPTS and the restart of docker in swarm-vagrant-manager.json aren't going to work for me.

Does this seem like a likely source of why I can't get Infrakit stood up and talking to the docker daemon?

YujiOshima commented 7 years ago

@dougbeauchene

Hi! In systemd, docker daemon option is define /lib/systemd/system/docker.service. So this maybe work for you.

"Init": [
          "docker swarm init --advertise-addr 192.168.2.200",
          "sed -i -e "s/ExecStart=\/usr\/bin\/dockerd -H fd:\/\//ExecStart=\/usr\/bin\/dockerd -H fd:\/\/ -H tcp:\/\/0.0.0.0:4243/g" /lib/systemd/system/docker.service",
          "systemctl daemon-reload",
          "service docker restart"
]

/etc/docker/daemon.json is also config of docker daemon. But options in the daemon.json is overridden by options in flag .

dougbeauchene commented 7 years ago

@YujiOshima Thank you for the response.

I had tried using /etc/docker/daemon.json but was unsuccessful. I assume, as you mention, it was being overridden.

I tried using your "Init" example above to update /lib/systemd/system/docker.service. First I did have to tweek the "sed...." to be valid json. I changed it to be...

"sed -i -e \"s/ExecStart=\/usr\/bin\/dockerd -H fd:\/\//ExecStart=\/usr\/bin\/dockerd -H fd:\/\/ -H tcp:\/\/0.0.0.0:4243/g\" /lib/systemd/system/docker.service"

When I ran through the startup, it still did not work.

/lib/systemd/system/docker.service is owned by root, thus can only be written to by root.

I am invoking infrakit group commit swarm-vagrant-manager.json manually with sudo but /lib/systemd/system/docker.service was not updated as part of that processing.

dougbeauchene commented 7 years ago

Don't all of these in swarm-vagrant-manager.json need to run under sudo?

docker swarm init --advertise-addr 192.168.2.200

sed -i -e "s/ExecStart=\/usr\/bin\/dockerd -H fd:\/\//ExecStart=\/usr\/bin\/dockerd -H fd:\/\/ -H tcp:\/\/0.0.0.0:4243/g" /lib/systemd/system/docker.service

systemctl daemon-reload

service docker restart
YujiOshima commented 7 years ago

@dougbeauchene Sorry for my typo..

Don't all of these in swarm-vagrant-manager.json need to run under sudo?

Init commands in vagrant instance plugin are vagrant Shell Provisioner script (boot.sh). https://github.com/docker/infrakit/blob/master/pkg/example/instance/vagrant/instance.go#L84 So the commands run as root.

I refactor the commands, it's worked for me. 

{
  "ID": "swarm-managers",
  "Properties": {
    "Allocation": {
      "LogicalIDs": ["192.168.2.200"]
    },
    "Instance": {
      "Plugin": "instance-vagrant",
      "Properties": {
        "Box": "ubuntu/xenial64"
      }
    },
    "Flavor": {
      "Plugin": "flavor-vanilla",
      "Properties": {
        "Init": [
          "curl -sSL https://get.docker.com/ | sh",
          "docker swarm init --advertise-addr 192.168.2.200",
          "sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service",
          "systemctl daemon-reload",
          "service docker restart"
        ]
      }
    }
  }
}
dougbeauchene commented 7 years ago

@YujiOshima

I replaced my swarm-vagrant-manager.json with the entire json you posted above.

{
  "ID": "swarm-managers",
  "Properties": {
    "Allocation": {
      "LogicalIDs": ["10.0.2.15"]
    },
    "Instance": {
      "Plugin": "instance-vagrant",
      "Properties": {
        "Box": "ubuntu/xenial64"
      }
    },
    "Flavor": {
      "Plugin": "flavor-vanilla",
      "Properties": {
        "Init": [
          "curl -sSL https://get.docker.com/ | sh",
          "docker swarm init --advertise-addr 10.0.2.15",
          "sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service",
          "systemctl daemon-reload",
          "service docker restart"
        ]
      }
    }
  }
}

I started all of the plugins, each in their own terminal session using these commands...

sudo workspace/src/github.com/docker/infrakit/build/infrakit-group-default
sudo workspace/src/github.com/docker/infrakit/build/infrakit-flavor-vanilla
sudo workspace/src/github.com/docker/infrakit/build/infrakit-instance-vagrant
sudo workspace/src/github.com/docker/infrakit/build/infrakit-flavor-swarm --host tcp://10.0.2.15:4243

I started the manager node...

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit group commit workspace/src/github.com/docker/infrakit/pkg/example/flavor/swarm/swarm-vagrant-manager.json
[sudo] password for <myuser>: 
Committed swarm-managers: Managing 1 instances

In the shell where I started the infrakit-group-default plugin I see...

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit-group-default
[sudo] password for <myuser>: 
INFO[0000] Listening at: /root/.infrakit/plugins/group  
INFO[0097] Committing group swarm-managers (pretend=false) 
INFO[0097] Logical ID 10.0.2.15 is missing, provisioning new instance 
ERRO[0426] Failed to provision: exit status 1

When I check on the contents of /lib/systemd/system/docker.service it does not appear that the ExecStart is updated...

$ sudo cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

When I try to list the swarm nodes, it fails to connect to the daemon...

$ sudo docker -H tcp://10.0.2.15:4243 node ls
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

As I mentioned in my post yesterday, when I ran the commands manually that are in swarm-vagrant-manager.json I had success finally getting connected to the Docker daemon...which is why I questioned if the commands needed to run under sudo.

Any advice you can offer would be appreciated.

Thanks Doug

YujiOshima commented 7 years ago

@dougbeauchene Is vagrant instance created? Please show me the detail log. try workspace/src/github.com/docker/infrakit/build/infrakit-group-default --log=5 and sudo workspace/src/github.com/docker/infrakit/build/infrakit-instance-vagrant --log=5

dougbeauchene commented 7 years ago

@YujiOshima

I added the --log=5 as you requested and ran the steps. Below are the output I see...

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit group commit workspace/src/github.com/docker/infrakit/pkg/example/flavor/swarm/swarm-vagrant-manager.json
[sudo] password for <myuserid>: 
Committed swarm-managers: Managing 1 instances
$ sudo workspace/src/github.com/docker/infrakit/build/infrakit-group-default --log=5
[sudo] password for <myuserid>: 
DEBU[0000] Opening: /root/.infrakit/plugins             
INFO[0000] Listening at: /root/.infrakit/plugins/group  
DEBU[0098] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 83
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":5577006791947779410} 
DEBU[0098] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Group","Version":"0.1.0"}]},"id":5577006791947779410}

DEBU[0098] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 613
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Group.CommitGroup","params":{"Spec":{"ID":"swarm-managers","Properties":{"Allocation":{"LogicalIDs":["10.0.2.15"]},"Instance":{"Plugin":"instance-vagrant","Properties":{"Box":"ubuntu/xenial64"}},"Flavor":{"Plugin":"flavor-vanilla","Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]}}}},"Pretend":false},"id":8674665223082153551} 
DEBU[0098] Opening: /root/.infrakit/plugins             
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/flavor-swarm 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/flavor-vanilla 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/group 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/instance-vagrant 
DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":1021172877274792647} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 99
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Flavor","Version":"0.1.0"}]},"id":1021172877274792647}

DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Flavor.Validate","params":{"Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]},"Allocation":{"Size":0,"LogicalIDs":["10.0.2.15"]}},"id":5007567516756375270} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 64
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"OK":true},"id":5007567516756375270}

DEBU[0098] Opening: /root/.infrakit/plugins             
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/flavor-swarm 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/flavor-vanilla 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/group 
DEBU[0098] Discovered plugin at /root/.infrakit/plugins/instance-vagrant 
DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":1085937180550891276} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 101
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Instance","Version":"0.1.0"}]},"id":1085937180550891276}

DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.Validate","params":{"Properties":{"Box":"ubuntu/xenial64"}},"id":924721567856795170} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 63
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"OK":true},"id":924721567856795170}

INFO[0098] Committing group swarm-managers (pretend=false) 
DEBU[0098] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"Details":"Managing 1 instances"},"id":8674665223082153551}

DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.DescribeInstances","params":{"Tags":{"infrakit.group":"swarm-managers"}},"id":6691302531857554516} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 72
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"Descriptions":[]},"id":6691302531857554516}

DEBU[0098] Found existing instances: []                 
INFO[0098] Logical ID 10.0.2.15 is missing, provisioning new instance 
DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Flavor.Prepare","params":{"Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]},"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"","LogicalID":"10.0.2.15","Attachments":null},"Allocation":{"Size":0,"LogicalIDs":["10.0.2.15"]}},"id":224881392039675580} 
DEBU[0098] Received response HTTP/1.1 200 OK
Content-Length: 529
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 14:22:14 GMT

{"jsonrpc":"2.0","result":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":224881392039675580}

DEBU[0098] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.Provision","params":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":187476663623441110} 
$ sudo workspace/src/github.com/docker/infrakit/build/infrakit-instance-vagrant --log=5
[sudo] password for <myuserid>: 
INFO[0000] Listening at: /root/.infrakit/plugins/instance-vagrant 
DEBU[0059] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 83
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":1085937180550891276} 
DEBU[0059] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Instance","Version":"0.1.0"}]},"id":1085937180550891276}

DEBU[0059] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 120
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.Validate","params":{"Properties":{"Box":"ubuntu/xenial64"}},"id":924721567856795170} 
DEBU[0059] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"OK":true},"id":924721567856795170}

DEBU[0059] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 134
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.DescribeInstances","params":{"Tags":{"infrakit.group":"swarm-managers"}},"id":6691302531857554516} 
DEBU[0059] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"Descriptions":[]},"id":6691302531857554516}

DEBU[0059] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 558
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.Provision","params":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":187476663623441110}

When I try to list node...

$ sudo docker -H tcp://10.0.2.15:4243 node ls
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

The docker config file does not appear to be updated...

$ sudo cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
dougbeauchene commented 7 years ago

@YujiOshima

In case it is helpful to you to see the logs from a somewhat successful (manual) series of steps, I'm providing the following.

I reset my enviornment. I started the plug-ins like above, with logging.

I ran these commands from swarm-vagrant-manager.json manually...

sudo curl -sSL https://get.docker.com/ | sh
sudo docker swarm init --advertise-addr 10.0.2.15
sudo sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service
sudo systemctl daemon-reload
sudo service docker restart

then I went ahead and ran this command...

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit group commit workspace/src/github.com/docker/infrakit/pkg/example/flavor/swarm/swarm-vagrant-manager.json
Committed swarm-managers: Managing 1 instances

The plugin-in log output is...

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit-group-default --log=5
DEBU[0000] Opening: /root/.infrakit/plugins             
INFO[0000] Listening at: /root/.infrakit/plugins/group  
DEBU[0241] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 83
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":5577006791947779410} 
DEBU[0241] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Group","Version":"0.1.0"}]},"id":5577006791947779410}

DEBU[0241] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 613
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Group.CommitGroup","params":{"Spec":{"ID":"swarm-managers","Properties":{"Allocation":{"LogicalIDs":["10.0.2.15"]},"Instance":{"Plugin":"instance-vagrant","Properties":{"Box":"ubuntu/xenial64"}},"Flavor":{"Plugin":"flavor-vanilla","Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]}}}},"Pretend":false},"id":8674665223082153551} 
DEBU[0241] Opening: /root/.infrakit/plugins             
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/flavor-swarm 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/flavor-vanilla 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/group 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/instance-vagrant 
DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":622070655531370019} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 98
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Flavor","Version":"0.1.0"}]},"id":622070655531370019}

DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Flavor.Validate","params":{"Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]},"Allocation":{"Size":0,"LogicalIDs":["10.0.2.15"]}},"id":733149503356615828} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 63
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"OK":true},"id":733149503356615828}

DEBU[0241] Opening: /root/.infrakit/plugins             
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/flavor-swarm 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/flavor-vanilla 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/group 
DEBU[0241] Discovered plugin at /root/.infrakit/plugins/instance-vagrant 
DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":7876419960494019292} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 101
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Instance","Version":"0.1.0"}]},"id":7876419960494019292}

DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.Validate","params":{"Properties":{"Box":"ubuntu/xenial64"}},"id":5985405036356110535} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 64
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"OK":true},"id":5985405036356110535}

INFO[0241] Committing group swarm-managers (pretend=false) 
DEBU[0241] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"Details":"Managing 1 instances"},"id":8674665223082153551}

DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.DescribeInstances","params":{"Tags":{"infrakit.group":"swarm-managers"}},"id":5467938796483798564} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 72
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"Descriptions":[]},"id":5467938796483798564}

DEBU[0241] Found existing instances: []                 
INFO[0241] Logical ID 10.0.2.15 is missing, provisioning new instance 
DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Flavor.Prepare","params":{"Properties":{"Init":["curl -sSL https://get.docker.com/ | sh","docker swarm init --advertise-addr 10.0.2.15","sed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service","systemctl daemon-reload","service docker restart"]},"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"","LogicalID":"10.0.2.15","Attachments":null},"Allocation":{"Size":0,"LogicalIDs":["10.0.2.15"]}},"id":464747708650094251} 
DEBU[0241] Received response HTTP/1.1 200 OK
Content-Length: 529
Content-Type: text/plain; charset=utf-8
Date: Wed, 14 Dec 2016 18:02:44 GMT

{"jsonrpc":"2.0","result":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":464747708650094251}

DEBU[0241] Sending request POST / HTTP/1.1
Host: a
Content-Type: application/json

{"jsonrpc":"2.0","method":"Instance.Provision","params":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":4527644784704883189} 

$ sudo workspace/src/github.com/docker/infrakit/build/infrakit-instance-vagrant --log=5
INFO[0000] Listening at: /root/.infrakit/plugins/instance-vagrant 
DEBU[0200] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 83
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Plugin.Implements","params":{},"id":7876419960494019292} 
DEBU[0200] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"APIs":[{"Name":"Instance","Version":"0.1.0"}]},"id":7876419960494019292}

DEBU[0200] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 121
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.Validate","params":{"Properties":{"Box":"ubuntu/xenial64"}},"id":5985405036356110535} 
DEBU[0200] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"OK":true},"id":5985405036356110535}

DEBU[0200] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 134
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.DescribeInstances","params":{"Tags":{"infrakit.group":"swarm-managers"}},"id":5467938796483798564} 
DEBU[0200] Sending response HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff

{"jsonrpc":"2.0","result":{"Descriptions":[]},"id":5467938796483798564}

DEBU[0200] Received request POST / HTTP/1.1
Host: a
Accept-Encoding: gzip
Content-Length: 559
Content-Type: application/json
User-Agent: Go-http-client/1.1

{"jsonrpc":"2.0","method":"Instance.Provision","params":{"Spec":{"Properties":{"Box":"ubuntu/xenial64"},"Tags":{"infrakit.config_sha":"z3TMOpTx34DUEu3reSGhTz7D0zM=","infrakit.group":"swarm-managers"},"Init":"curl -sSL https://get.docker.com/ | sh\ndocker swarm init --advertise-addr 10.0.2.15\nsed -i -e 's@ExecStart=/usr/bin/dockerd -H fd://@ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:4243@g' /lib/systemd/system/docker.service\nsystemctl daemon-reload\nservice docker restart","LogicalID":"10.0.2.15","Attachments":null}},"id":4527644784704883189} 
$ sudo docker -H tcp://10.0.2.15:4243 node ls
ID                           HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS
c50yk8q3mf7yl2emgkad3fu6a *  manager1  Ready   Active        Leader