docker-archive / dockercloud-haproxy

HAproxy image that autoreconfigures itself when used in Docker Cloud
https://cloud.docker.com/
651 stars 187 forks source link

Cannot get it to run in docker swarm #226

Closed levino closed 6 years ago

levino commented 6 years ago

I try to set up load balancing for a docker swarm stack. Here is the logs from haproxy:

INFO:haproxy:dockercloud/haproxy 1.6.7 is running outside Docker Cloud
INFO:haproxy:Haproxy is running using legacy link, loading HAProxy definition from environment variables: unable to connect to docker daemon UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
INFO:haproxy:dockercloud/haproxy PID: 7
INFO:haproxy:=> Add task: Initial start - Legacy Mode
INFO:haproxy:=> Executing task: Initial start - Legacy Mode
INFO:haproxy:==========BEGIN==========
INFO:haproxy:SSL certificates are updated
INFO:haproxy:HAProxy configuration:
global
  log 127.0.0.1 local0
  log 127.0.0.1 local1 notice
  log-send-hostname
  maxconn 4096
  pidfile /var/run/haproxy.pid
  user haproxy
  group haproxy
  daemon
  stats socket /var/run/haproxy.stats level admin
  ssl-default-bind-options no-sslv3
  ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DHE-DSS-AES128-SHA:DES-CBC3-SHA
defaults
  balance roundrobin

obal
  mode http
  option redispatch
  option httplog
  option dontlognull
  option forwardfor
  timeout connect 5000
  timeout client 50000
  timeout server 50000

tats

1936
  mode http
  stats enable
  timeout connect 10s
  timeout client 1m
  timeout server 1m
  stats hide-version
  stats realm Haproxy\ Statistics
  stats uri /
  stats auth stats:stats
INFO:haproxy:Launching HAProxy
INFO:haproxy:HAProxy has been launched(PID: 10)
INFO:haproxy:===========END===========

Here is my loadbalancer stackfile:

version: '3.3'
services:
  lb:
    image: dockercloud/haproxy:1.6.7
    environment:
      DEFAULT_SSL_CERT: "secret"
    ports:
      - 80:80
      - 443:443
    networks:
      - proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      placement:
        constraints:
          - "node.role == manager"
networks:
  proxy:
    external: true

And here is the service I want to connect to:

version: "3.3"
services:
  mongo:
    image: mongo:3
    environment:
      - AUTH=no
      - LOG_LEVEL=debug
    volumes:
      - /storage/blooks/mongo:/data/db
    hostname: mongo
  redis:
    image: redis:3
  dashboard:
    image: 'blooks/dashboard:v0.1.5'
    environment:
      - MONGO_URL=mongodb://mongo/blooks
      - REDIS_URL=redis://redis
      - VIRTUAL_HOST=app.blooks.io
      - FORCE_SSL=true
      - ROOT_URL=https://app.blooks.io
      - SERVICE_PORTS="3000"
    networks:
      - default
      - proxy
    ports:
      - 8080:3000
networks:
  default:
  proxy:
    external: true

When I try to access http://app.blooks.io nothing works. "Refused to connect". When I access http://app.blooks.io:8080 it works. What am I doing wrong?

The swarm only has one node.

levino commented 6 years ago

This has been fixed. The problem was that I secured the docker daemon on the server to access it remotely. So I changed my /etc/docker/daemon.json to this

{
  "tlsverify": true,
  "tlscacert": "/var/docker/ca.pem",
  "tlscert": "/var/docker/server-cert.pem",
  "tlskey": "/var/docker/server-key.pem",
  "hosts": ["tcp://0.0.0.0:2376"]
}

But dockercloud/haproxy wants to communicate via the unix socket at /var/run/docker.sock which my dockerd would not be using to any more. So changing the file to the following worked:

{
  "tlsverify": true,
  "tlscacert": "/var/docker/ca.pem",
  "tlscert": "/var/docker/server-cert.pem",
  "tlskey": "/var/docker/server-key.pem",
  "hosts": ["tcp://0.0.0.0:2376","unix:///var/run/docker.sock"]
}

Fortunately the connection on the unix port does not require tls authentication. I wonder how I would set that in dockercloud/haproxy :relieved: