Enhancement: Add Docker support

[GoogleCodeExporter]

The running of Docker within a Core node will allow much more extensibility to 
the services that Core can run within them.

This allows network applications and protocols to be packaged and run on any 
node easily.

I attach an example service that will add a new group to the services. This 
will have a service called Docker which will just start the docker service 
within the node but not run anything. It will also scan all docker images on 
the host machine. If any are tagged with 'core' then they will be added as a 
service to the Docker group. The image will then be auto run if that service is 
selected.

This requires a recent version of Docker. I am using a PPA on Ubuntu and have 
version 1.2.0. The version in the standard Ubuntu repo is to old for this 
purpose (we need --net host).

It also require docker-py (https://pypi.python.org/pypi/docker-py) which can be 
installed with 'pip install docker-py'. This is used to interface with Docker 
from the python service.

An example use case is to pull an image from Docker.com. I have uploaded a test 
image for this purpose:

sudo docker pull stuartmarsden/multicastping

This downloads an image which is based on Ubuntu 14.04 with python and twisted. 
It runs a simple program that sends a multicast ping and listens and records 
any it receives.

In order for this to show up as a docker service it must be tagged with core. 
Find out the id by running 'sudo docker images'. You should see all installed 
images and the one you want looks like this:

stuartmarsden/multicastping    latest              4833487e66d2        20 hours 
ago        487 MB

The id will be different on your machine so use it in the following command:

sudo docker tag 4833487e66d2 stuartmarsden/multicastping:core

This image will be listed in the services after we restart the core-daemon: 

sudo service core-daemon restart

You can set up a simple network with a number of PCs connected to a switch. Set 
the stuartmarsden/multicastping service for all the PCs. When started they will 
all begin sending Multicast pings. 

In order to see what is happening you can go in to the terminal of a node and 
look at the docker log. Easy shorthand is:

docker logs $(docker ps -q)

Which just shows the log of the running docker container (usually just one per 
node). I have added this as an observer node to my setup:
Name: docker logs
Command: bash -c 'docker logs $(docker ps -q) | tail -20'

So I can just hover over to see the log which looks like this:

Datagram 'Client: Ping' received from ('10.0.0.20', 8005)
Datagram 'Client: Ping' received from ('10.0.5.21', 8005)
Datagram 'Client: Ping' received from ('10.0.3.20', 8005)
Datagram 'Client: Ping' received from ('10.0.4.20', 8005)
Datagram 'Client: Ping' received from ('10.0.4.20', 8005)
Datagram 'Client: Ping' received from ('10.0.1.21', 8005)
Datagram 'Client: Ping' received from ('10.0.4.21', 8005)
Datagram 'Client: Ping' received from ('10.0.4.21', 8005)
Datagram 'Client: Ping' received from ('10.0.5.20', 8005)
Datagram 'Client: Ping' received from ('10.0.0.21', 8005)
Datagram 'Client: Ping' received from ('10.0.3.21', 8005)
Datagram 'Client: Ping' received from ('10.0.0.20', 8005)
Datagram 'Client: Ping' received from ('10.0.5.21', 8005)
Datagram 'Client: Ping' received from ('10.0.3.20', 8005)
Datagram 'Client: Ping' received from ('10.0.4.20', 8005)
Datagram 'Client: Ping' received from ('10.0.4.20', 8005)
Datagram 'Client: Ping' received from ('10.0.1.21', 8005)
Datagram 'Client: Ping' received from ('10.0.4.21', 8005)
Datagram 'Client: Ping' received from ('10.0.4.21', 8005)
Datagram 'Client: Ping' received from ('10.0.5.20', 8005)

Limitations:

1. Docker images must be downloaded on the host as usually a Core node does not 
have access to the internet.
2. Each node isolates running containers (keeps things simple)
3. Recent version of docker needed so that --net host can be used. This does 
not further abstract the network within a node and allows multicast which is 
not enabled within Docker containers at the moment.
4. The core-daemon must be restarted for new images to show up.
5. A Docker-daemon is run within each node but the images are shared. This does 
mean that the daemon attempts to access an SQLlite database within the host. At 
startup all the nodes will try to access this and it will be locked for most 
due to contention. The service just does a hackish wait for 1 second and retry. 
This means all the docker containers can take a while to come up depending on 
how many nodes you have.

Original issue reported on code.google.com by stuartma...@gmail.com on 3 Sep 2014 at 9:53

Attachments:

• dockerService.py
• CoreDocker.png

[GoogleCodeExporter]

comments on Python service:
- setting name server in resolv.conf will affect the DNS resolution on the host?
- need try/except blocks around import of docker module so it is not required 
for all users

Original comment by ahrenh...@gmail.com on 30 Oct 2014 at 8:03

• Changed state: Accepted
• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

This issue was closed by revision r603.

Original comment by ahrenh...@gmail.com on 31 Oct 2014 at 4:45

• Changed state: Fixed