Docker 1.12 Swarm Mode

[sgillespie]

The new swarm mode doesn't appear to expose running containers/services like Docker Swarm previous did. For that reason, it would seem that this project is not available to me. Are there any plans to account for that?

[mrlesmithjr]

Confirming I am also seeing the same results with spinning up an ELK Stack deployment using docker-compose and using docker-gen with HAProxy. Docker version and Docker Swarm version info:

Client:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 21:04:48 2016
 OS/Arch:      darwin/amd64
 Experimental: true

Server:
 Version:      swarm/1.2.4
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   5d5f7f0
 Built:        Thu Jul 28 19:52:54 UTC 2016
 OS/Arch:      linux/amd64

Below is my docker-compose.yml:

version: '2'
services:
  haproxy:
    depends_on:
      - elasticsearch
      - redis
      - elasticsearch_master
      - kibana
      - logstash_pre_processor
      - logstash_processor
    build: ./HAProxy
    image: elk-haproxy
    # links:
    #   - elasticsearch
    #   - elasticsearch_master
    #   - kibana
    #   - logstash_pre_processor
    #   - logstash_processor
    #   - redis
    networks:
      - front
      - back
    ports:
      - "5044:5044"
      - "5601:5601"
      - "6379:6379"
      - "9090:9090"
      - "9200:9200"
      - "9300:9300"
      - "10514:10514"
      - "10514:10514/udp"
    volumes:
      - "/var/run/docker.sock:/tmp/docker.sock:ro"
    restart: always
    environment:
      - "BALANCE=roundrobin"
      - "ES_BACKEND=elasticsearch"
      - "ES_BACKEND_PORT=9200"
      - "ES_MASTER_BACKEND=elasticsearch_master"
      - "ES_MASTER_BACKEND_PORT=9300"
      - "KIBANA_BACKEND=kibana"
      - "KIBANA_BACKEND_PORT=5601"
      - "LOGSTASH_PRE_PROCESSOR_BACKEND=logstash_pre_processor"
      - "PROJECT=dockeransibleelkstack"
      - "REDIS_BACKEND=redis"
      - "REDIS_BACKEND_PORT=6379"
      - "SERVICE=haproxy"

  elasticsearch_master:
    build: ./Elasticsearch
    image: elk-elasticsearch
    command: "elasticsearch -Des.node.master=true -Des.node.data=false -Des.discovery.zen.ping.unicast.hosts=haproxy"
    restart: always
    networks:
      - back

  elasticsearch:
    # depends_on:
    #   - elasticsearch_master
    image: elk-elasticsearch
    command: "elasticsearch -Des.node.master=false -Des.discovery.zen.ping.unicast.hosts=haproxy"
    # links:
    #   - elasticsearch_master
    volumes:
      - "./.data:/usr/share/elasticsearch/data"
    restart: always
    networks:
      - back

  kibana:
    depends_on:
      - elasticsearch
    build: ./Kibana
    image: elk-kibana
    restart: always
    networks:
      - back

  logstash_pre_processor:
    build: ./Logstash_pre_processor
    image: elk-logstash-pre-processor
    restart: always
    networks:
      - back

  logstash_processor:
    depends_on:
      - elasticsearch
    build: ./Logstash_processor
    image: elk-logstash-processor
    restart: always
    networks:
      - back

  redis:
    image: mrlesmithjr/redis:latest
    restart: always
    volumes:
      - "./.data:/data"
    networks:
      - back

networks:
  front:
    driver: overlay
  back:
    driver: overlay

# networks:
#   default:
#     external:
#       name: my-net

Thoughts!

Link to my GitHub repo with the above config as well under branch swarm-cluster. https://github.com/mrlesmithjr/docker-ansible-elkstack/tree/swarm-cluster

[giammbo]

i'm using nginx reverse proxy, and i have 2 node of docker swarm, when i create the container with the environment VIRTUAL_HOST and the network "overlay" the docker-gen doesn't take the ip, i recieve only:

upstream pluto3.com { server :80; }

[vincentfretin]

@giammy2290 you may find a solution in #169 I now have a bridge network created with docker-compose version 2 format, I myself changed my template to use this:

{{ range $host, $containers := groupBy $ "Env.VIRTUAL_HOST" }}                  
upstream {{ $host }} {                                                          
{{ range $index, $value := $containers }}                                       
    {{ $network := index $value.Networks 0 }}                                   
    {{ with $address := index $value.Addresses 0 }}                             
    server {{ $network.IP }}:{{ $address.Port }};                               
    {{ end }}                                                                   
{{ end }}                                                                       
} 

[giammbo]

@vincentfretin yes this works for me, but if i use 2 or more docker node (swarm cluster) docker-gen doesn't work (i have set the endpoint to the docker master node api)

[s4s0l]

@giammy2290 for that reason i use socat on manager nodes and run docker-gen container attached to it not to var/run/docker.sock. It's a good idea to secure such socat endpoint. then docker-gen work regardles of where it is run.

[lvnilesh]

@s4s0l Appreciate if you can please provide a socat example to make it work with multi-node swarm cluster

[s4s0l]

I'm working on it at the moment. What I've got so far is a sockat running on manager node host OS. I'm working on some docker image so this socat can be run in 'global' mode in swarm. I'll put it somewhere in github. It'll probably take me some time to do it, but it is something like (first part is in groovy but i think its reverse engeneerable):

def generateKeyPair(String cn,
                        List<String> ips, List<String> dns,
                        String keyOutFile, String certOutFile) {
        LOGGER.info("openssl: generating key $keyOutFile for ${cn} (${ips.join(",")})")
        new File(keyOutFile).getParentFile().mkdirs()
        new File(certOutFile).getParentFile().mkdirs()
        File tmp = File.createTempFile("sslcfg", "cnf");
        tmp.text = """
${OpenSslWrapper.getResource('/openssl.cnf').text}
[ v3_xxx ]
basicConstraints = critical, CA:true
keyUsage=critical, digitalSignature,keyEncipherment,keyAgreement,dataEncipherment,keyCertSign
extendedKeyUsage=critical,serverAuth,clientAuth,codeSigning
subjectAltName=${(ips.collect { "IP:$it" } + dns.collect { "DNS:$it" }).join(",")}
"""
        exec.executeForOutput("""
            req -newkey rsa:4096 -nodes -sha256 -x509 -days 365 -subj /CN=${cn}
            -extensions v3_xxx -config ${tmp.absolutePath}
            -keyout ${keyOutFile} 
            -out ${certOutFile}
        """.replace("\n", ""))

    }
generateKeyPair("localhost", [], ["metadata", "localhost"] , "$path/server.key","$path/server.crt")
generateKeyPair("localhost", [], ["metadata", "localhost"] , "$path/client.key","$path/client.crt")

I'll make it more portable later, the point is generate certs and keys for the server (socat) and client (docker-gen), see http://www.dest-unreach.org/socat/doc/socat-openssltunnel.html and https://youremindmeofmymother.com/2015/08/21/socat-ssl-ssl-routinesssl3_check_cert_and_algorithmdh-key-too-small/ then:

cat server.key server.crt > server.pem
cat client.key client.crt > client.pem
#this one will take some sirious time it might be worth changing 4096 to something more realistic
openssl dhparam -out dhparams.pem 4096
cat dhparams.pem >> server.pem

The whole point is to start socat like this:

socat -d -d openssl-listen:3333,fork,reuseaddr,cert=$(pwd)/server.pem,cafile=$(pwd)/client.crt unix-connect:/var/run/docker.sock
#and then you can test it like 
curl  --cacert server.crt --cert-type pem --cert client.pem https://localhost:3333/v1.26/_ping
#and finally parameters for docker-gen:
./docker-gen --config ./docker-gen.ini -tlscacert server.crt -tlscert client.crt -tlskey client.key -watch -endpoint tcp://localhost:3333

As I said I'm working on some more shareable form of it, but decided to paste it here as maybe you guys would have some comments and save me from doing something wrong;)

[s4s0l]

Ok, the most important is https://github.com/jwilder/docker-gen/pull/239 anyway. Forgot the obvious.

[s4s0l]

@lvnilesh In my fork is full working example https://github.com/s4s0l/docker-gen.

I don't like it because it pulls for changes constantly (as swarm mode events are not included in /events docker api). I also dont like docker-gen is crushing when socat endpoint is unavailable and swarm restarts it until it becomes...but i works and in some circumstances may be the only way to hack around docker.

I've closed my PR as it generally is workaround for docker-gen which itself is kid of workaround so I convinced myself that it would do harm to docker-gen codebase;) But if @jwilder thins its ok, I can prepare it once again.

[lvnilesh]

Thanks @s4s0l

[A34]

There is a working solution: https://hub.docker.com/r/helder/docker-gen/

[arefaslani]

Checkout my image. It's written in javascript and will be developed easily. Pull requests are welcome 🙂