Powerstrip fails against swarm

aanm commented 9 years ago

I know this is too much experimental but it should work since docker-swarm has almost the same endpoints has docker

CONSOLE 1:

mkdir -p $HOME/powerstrip-swarm
cat > $HOME/powerstrip-swarm/adapters.yml << EOF
version: 1
endpoints:
  "POST /*/containers/create":
    pre: [debug]
    post: [debug]
adapters:
  debug: http://debug/extension
EOF

id=$(sudo docker run swarm create)
docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery=token://$id swarm-master
docker-machine create -d virtualbox --swarm --swarm-discovery=token://$id swarm-01
docker-machine create -d virtualbox --swarm --swarm-discovery=token://$id swarm-02

sudo docker run -d --name powerstrip-debug --expose 80 binocarlos/powerstrip-debug

__OLD_DOCKER_TLS_VERIFY=$DOCKER_TLS_VERIFY
__OLD_DOCKER_CERT_PATH=$DOCKER_CERT_PATH
__OLD_DOCKER_HOST=$DOCKER_HOST
$(docker-machine env --swarm swarm-master)
__SWARM_DOCKER_TLS_VERIFY=$DOCKER_TLS_VERIFY
__SWARM_DOCKER_CERT_PATH=$DOCKER_CERT_PATH
__SWARM_DOCKER_HOST=$DOCKER_HOST
DOCKER_TLS_VERIFY=$__OLD_DOCKER_TLS_VERIFY
DOCKER_CERT_PATH=$__OLD_DOCKER_CERT_PATH
DOCKER_HOST=$__OLD_DOCKER_HOST
sudo docker run -d --name powerstrip \
         -e DOCKER_TLS_VERIFY=$__SWARM_DOCKER_TLS_VERIFY \
         -e DOCKER_CERT_PATH=/swarm \
         -e DOCKER_HOST=$__SWARM_DOCKER_HOST \
         -v $HOME/powerstrip-swarm/adapters.yml:/etc/powerstrip/adapters.yml \
         -v $__SWARM_DOCKER_CERT_PATH:/swarm \
         --link powerstrip-debug:debug \
         -p 2375:2375 \
         clusterhq/powerstrip:v0.0.1

CONSOLE 2:

[aanm@localhost ~]$ DOCKER_HOST=localhost:2375 docker run busybox /bin/echo hello
FATA[0000] EOF

powerstrip-debug logs:

[aanm@localhost ~]$ sudo docker logs powerstrip-debug
server listening on port: 80

-------------------------------------------

Version:          1
Type:             pre-hook
Request:

    Method:      POST
    URL:         /v1.18/containers/create

{
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "Memory": 0,
    "MemorySwap": 0,
    "CpuShares": 0,
    "Cpuset": "",
    "AttachStdin": false,
    "AttachStdout": true,
    "AttachStderr": true,
    "PortSpecs": null,
    "ExposedPorts": {},
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [],
    "Cmd": [
        "/bin/echo",
        "hello"
    ],
    "Image": "busybox",
    "Volumes": {},
    "WorkingDir": "",
    "Entrypoint": null,
    "NetworkDisabled": false,
    "MacAddress": "",
    "OnBuild": null,
    "Labels": {},
    "HostConfig": {
        "Binds": null,
        "ContainerIDFile": "",
        "LxcConf": [],
        "Memory": 0,
        "MemorySwap": 0,
        "CpuShares": 0,
        "CpusetCpus": "",
        "Privileged": false,
        "PortBindings": {},
        "Links": null,
        "PublishAllPorts": false,
        "Dns": null,
        "DnsSearch": null,
        "ExtraHosts": null,
        "VolumesFrom": null,
        "Devices": [],
        "NetworkMode": "bridge",
        "IpcMode": "",
        "PidMode": "",
        "CapAdd": null,
        "CapDrop": null,
        "RestartPolicy": {
            "Name": "no",
            "MaximumRetryCount": 0
        },
        "SecurityOpt": null,
        "ReadonlyRootfs": false,
        "Ulimits": null,
        "LogConfig": {
            "Type": "",
            "Config": null
        },
        "CgroupParent": ""
    }
}

-------------------------------------------

Version:          1
Type:             post-hook
Request:

    Method:      POST
    URL:         /v1.18/containers/create

{
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "Memory": 0,
    "MemorySwap": 0,
    "CpuShares": 0,
    "Cpuset": "",
    "AttachStdin": false,
    "AttachStdout": true,
    "AttachStderr": true,
    "PortSpecs": null,
    "ExposedPorts": {},
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [],
    "Cmd": [
        "/bin/echo",
        "hello"
    ],
    "Image": "busybox",
    "Volumes": {},
    "WorkingDir": "",
    "Entrypoint": null,
    "NetworkDisabled": false,
    "MacAddress": "",
    "OnBuild": null,
    "Labels": {},
    "HostConfig": {
        "Binds": null,
        "ContainerIDFile": "",
        "LxcConf": [],
        "Memory": 0,
        "MemorySwap": 0,
        "CpuShares": 0,
        "CpusetCpus": "",
        "Privileged": false,
        "PortBindings": {},
        "Links": null,
        "PublishAllPorts": false,
        "Dns": null,
        "DnsSearch": null,
        "ExtraHosts": null,
        "VolumesFrom": null,
        "Devices": [],
        "NetworkMode": "bridge",
        "IpcMode": "",
        "PidMode": "",
        "CapAdd": null,
        "CapDrop": null,
        "RestartPolicy": {
            "Name": "no",
            "MaximumRetryCount": 0
        },
        "SecurityOpt": null,
        "ReadonlyRootfs": false,
        "Ulimits": null,
        "LogConfig": {
            "Type": "",
            "Config": null
        },
        "CgroupParent": ""
    }
}

Response:

    ContentType: null
    Code:        200

P.S. I make sure busybox was pulled in to the 2 nodes More-info: https://github.com/docker/swarm/issues/619

binocarlos commented 9 years ago

Can I ask what happens when you run your example busybox but attach stdin?

$ DOCKER_HOST=localhost:2375 docker run -i busybox /bin/echo hello

We were seeing this problem in v0.0.1 where when you try to attach to a container - it would fail silently with no content. The unix-socket branch here does not have this problem but you have to bind powerstrip to a unix socket.

The error you are seeing looks like it could be that problem.

aanm commented 9 years ago

@binocarlos Same thing happens. Also I've tried to run, through a docker client previously installed into powerstrip, and worked. I have to confirm if it works with this specific example.

aanm commented 9 years ago

Yep, I can confirm that it works with a "normal" docker client inside powerstrip

[aanm@localhost ~]$ sudo docker exec -ti powerstrip bash
root@9ca988eb0c33:/app# apt-get update && apt-get install wget &&  wget -qO- https://get.docker.com/ | sh
# After being installed....
root@9ca988eb0c33:/app# docker run busybox /bin/echo hello
hello
root@9ca988eb0c33:/app# docker ps -a
CONTAINER ID        IMAGE                       COMMAND                CREATED             STATUS                              PORTS                                     NAMES
6e88aea46692        busybox:buildroot-2014.02   "/bin/echo hello"      13 seconds ago      Exited (0) Less than a second ago                                             swarm-02/elated_curie             
7e5ca8eb1762        swarm:latest                "/swarm join --addr    About an hour ago   Up About an hour                    2375/tcp                                  swarm-02/swarm-agent              
f26f57f321ae        swarm:latest                "/swarm join --addr    About an hour ago   Up About an hour                    2375/tcp                                  swarm-01/swarm-agent              
fa0d2c0c5a29        swarm:latest                "/swarm join --addr    About an hour ago   Up About an hour                    2375/tcp                                  swarm-master/swarm-agent          
f51ef437d200        swarm:latest                "/swarm manage --tls   About an hour ago   Up About an hour                    2375/tcp, 192.168.99.100:3376->3376/tcp   swarm-master/swarm-agent-master

aanm commented 9 years ago

@binocarlos argh... the problem is this:

For now, Powerstrip does not support TLS, but given that it should only be used for prototyping in local development environments, that's OK.

And docker-machine only runs with TLS.

Is there any ETA for TLS on powerstrip?

binocarlos commented 9 years ago

@aanm hey...

So - there are no plans to support TLS in Powerstrip in the near future - we have a bunch of other stuff to focus on. That said - if a PR landed with this feature it would get serious consideration :-)

You could set DOCKER_TLS_VERIFY=0 or use a TLS terminator as a work-around.

I have a question - I notice you are using Powerstrip in front of the swarm master. This is a novel way to do it - the normal pattern is to have powerstrip installed on each docker host.

Here is a (simplified) diagram from the recent blog post we did demonstrating powerstrip and swarm together:

          docker client

                |

           swarm deamon

           /          \

       node1          node2

         |              |

    powerstrip       powerstrip

        |               |

      docker          docker

and here is a diagram of what seems to be happening in your setup:

           docker client

                |

            powerstrip

                |

           swarm deamon

           /          \

       node1          node2

        |               |

      docker          docker

What are the powerstrip adapters you intend to use in this setup?

acknowledged that docker-machine uses TLS as it's default setup so there will some degree of manual setup in order to work-around the TLS issue.

aanm commented 9 years ago

Hmm I don't know why but I thought the nodes would be all equal but they could or couldn't be... So... why not on both sides? ;-)

          docker client
                |
           powerstrip 
                |

           swarm deamon

           /          \

       node1          node2

         |              |

    powerstrip       powerstrip

        |               |

      docker          docker

binocarlos commented 9 years ago

Hey - so interesting setup :-)

In theory that would work - however, what adapters would make sense in the upper-most powerstrip?

aanm commented 9 years ago

That's what I'm still trying to figure it out :-)

Pensu commented 9 years ago

@aanm Were you able to make this setup work?

    docker client

                |

            powerstrip

                |

           swarm deamon

           /          \

       node1          node2

        |               |

      docker          docker

aanm commented 9 years ago

@Pensu I think so if the docker client isn't docker machine

binocarlos commented 9 years ago

@Pensu I think that setup would depend heavily on what Powerstrip adapters you were trying to use.

For example - if using powerstrip-weave - it would not work because the adapter would be running on a proxy node not the nodes that Docker itself is running on.

However - if you were wanting to change all containers to have extra environment variables - i.e. something that just changed the JSON body of the call to /containers/create and did not require software to be installed on the proxy machine - then it would work.

What are the adapters you are wanting to run in this setup?

Pensu commented 9 years ago

@binocarlos ok, I was actually trying to add an authentication mechanism(or adapter?) to swarm, just to try it out. Btw I was able to make powerstrip work with this setup and as @aanm said, not with docker-machine but with the development setup.

ClusterHQ / powerstrip

Powerstrip fails against swarm #80