Struggling to make it work

[sslash]

Struggling to set it up, here's my stackfile (tutum specific):

letsencrypt:
  image: 'levino/letsencrypt-cert-service:latest'
  environment:
    - 'CERT_DOMAINES=domain.guide,certs.domain.guide'
    - CERT_EMAIL=team@domain.guide
    - CERT_SERVICE_PASSWORD=secretsecret
    - CERT_SERVICE_PRIVATE=true
    - CERT_SERVICE_USERNAME=myuname
    - EXCLUDE_PORTS=443
    - 'VIRTUAL_HOST=http://*/.well-known/*,https://certs.domain.guide,http://certs.domain.guide'
    - VIRTUAL_HOST_WEIGHT=1
  volumes:
    - /etc/letsencrypt
load-balancer-staging:
  image: 'tutum/haproxy:latest'
  links:
    - node-blue
    - letsencrypt
  ports:
    - '80:80'
    - '443:443'
  roles:
    - global
node-blue:
  image: 'webapp-image'
  environment:
    - NODE_ENV=production
    - SERVER=true
    - 'VIRTUAL_HOST=domain.guide'
  ports:
    - '8000'

[levino]

CERT_DOMAINS not CERT_DOMAINES

Can you paste logs from the cert-service container?

[sslash]

letsencrypt-1 | 2016-02-11T19:13:11.769031599Z npm info it worked if it ends with ok letsencrypt-1 | 2016-02-11T19:13:11.769532442Z npm info using npm@2.14.12 letsencrypt-1 | 2016-02-11T19:13:11.770303582Z npm info using node@v4.2.4 letsencrypt-1 | 2016-02-11T19:13:12.085206182Z npm info prestart letsencrypt-cert-service@0.1.0 letsencrypt-1 | 2016-02-11T19:13:12.090421271Z npm info start letsencrypt-cert-service@0.1.0 letsencrypt-1 | 2016-02-11T19:13:12.094726047Z letsencrypt-1 | 2016-02-11T19:13:12.094747280Z > letsencrypt-cert-service@0.1.0 start /usr/src/app letsencrypt-1 | 2016-02-11T19:13:12.094758678Z > node ./start.js letsencrypt-1 | 2016-02-11T19:13:12.094768247Z letsencrypt-1 | 2016-02-11T19:13:12.613974975Z {"name":"LetsEncrypt Cert Service","hostname":"letsencrypt-1","pid":25,"level":30,"msg":"Succeeded to start cert service","time":"2016-02-11T19:13:12.611Z","v":0}

[levino]

Did you change above env var? Still an issue?

[sslash]

jupp. getting 503 service unavailable when I try to go to haproxy url

[levino]

You should get a 404 from node-blue...

[levino]

Where is this deployed? Not under "domain.guide"

[sslash]

node-blue responds normally. Haproxy still gives me this: "503 Service Unavailable No server is available to handle this request." Its not deployed under that domain, because this is just a sandbox environment I sat up to test if it works. When I tried it in production, I got the 503, so I would like to be a bit more confident it will work before I deploy the setup in production. At some point I'll point routes.guide to the sandbox environment (temporary to verify it works) but then I want to get rid of the 503 first

[levino]

I don't follow.

When you go to "http://domain.guide" node-blue answers?

When you go to "http://domain.guide/welcome" node-blue answers?

When you go to "http://domain.guide/.well-known/check" you a get a 503?

The node-blue should also answer in the last case if the cert service fails to answer.

[sslash]

Ah no no, every url always responds with 503. Thats why I took it down, and now I am trying the same setup on a different cluster. Still the haproxy url gives 503

[levino]

Okay. Tell me when you get "http://domain.guide" to work. This will mean that the loadbalancer - blue-node setup is working. Explaining how to achieve this is out of scope here.

I would guess that you are querying the ip where you deployed this like "http://123.142.24.11" but NOT "http://domain.guide". Querying the IP will NOT work. How should haproxy know where you wanna go? A workaround is to put the "domain.guide" host in your /etc/hosts to target to the above IP. Then you can go in the browser to "http://domain.guide" wich will be resolved to the aforementioned IP, WHILE telling the server that you are actually looking for "http://domain.guide".

BTW: The service can be risk added risk free to any production environment that has a working haproxy setup. Traffic will continue to be correctly routed. Your 503s above have nothing to do with adding a containter with my image. It is an issue of the overall setup.

[sslash]

My haproxy-express setup works well without this service. However when I add a link from haproxy to letsencrypt, and set VIRTUAL_HOST in the express definition, haproxy responds with a 503 (hitting the express server still responds normally)

[levino]

So just to clarify, the following works fine?

load-balancer-staging:
  image: 'tutum/haproxy:latest'
  links:
    - node-blue
  ports:
    - '80:80'
    - '443:443'
  roles:
    - global
node-blue:
  image: 'webapp-image'
  environment:
    - NODE_ENV=production
    - SERVER=true
    - 'VIRTUAL_HOST=domain.guide'
  ports:
    - '8000'

Could you paste the complete logs of the haproxy when you link the the cert service? It should show an indication of that it acknowledges to route traffic to the cert service to certs.domain.guide.

Could you deploy this stack under a real hostname / subdomain like "test.domain.guide"?

[sslash]

This is interesting. The setup you shows, gives 503 (though hitting node-blue directly works). However, if I remove 'VIRTUAL_HOST=domain.guide' in node-blue, haproxy works fine again. (this is without adding cert service btw)

[levino]

Okay. You definitely need to host this on a proper domain. What exactly do you put in your browser to reach the haproxy - node-blue setup in this case?

[sslash]

trying both domain.guide, and http://load-balancer-st.staging.b8f1bf8d.svc.dockerapp.io. The latter works actually, if I add http://load-balancer-st.staging.b8f1bf8d.svc.dockerapp.io to VIRTUAL_HOSTS.

[levino]

Of course. As I said. What you put in the "url" field in your browser will be sent to haproxy as "this user is looking for http://domain.guide. do you know someone who is called "domain.guide"?" If you go to "http://load-balancer-st.staging.b8f1bf8d.svc.dockerapp.io" haproxy will say (in the above setup): "http://load-balancer-st.staging.b8f1bf8d.svc.dockerapp.io" was never known here, go away, aka 503.

So just create a subdomain test.domain.guide and host everything there and get it running.

The reason why it works without the "VIRTUAL_HOST" in blue-node is, that haproxy then just routes it all to the one backend it finds, but this is actually a bad default behaviour.

[sslash]

Progress. Made the 503 go away after adding VIRTUAL_PORT=8000 to express. So new problem: Hitting /makecert gives this error in the letsencrypt logs: letsencrypt-1 | 2016-02-18T08:42:23.708833253Z {"name":"LetsEncrypt Cert Service","hostname":"letsencrypt-1","pid":16,"level":50,"msg":"exec error: Error: Command failed: ./create-cert.sh\ndebconf: delaying package configuration, since apt-utils is not installed\nE: Sub-process /usr/bin/dpkg returned an error code (1)\n","time":"2016-02-18T08:42:23.307Z","v":0} letsencrypt-1 | 2016-02-18T08:44:35.107900465Z {"name":"LetsEncrypt Cert Service","hostname":"letsencrypt-1","pid":16,"level":30,"msg":"Restarted webservers","time":"2016-02-18T08:44:35.089Z","v":0}

both the check and status calls (as described in the usage section) worked fine

[levino]

The last message indicates success. Did you hit the endpoint twice or only once? The timestamps are 2 minutes apart from each other... The webservers should only be restarted when the certificates have been generated. Anything in the /etc/letsencrypt folders?

[sslash]

Yup, there's: accounts archive csr keys live renewal

[levino]

So it worked. Unfortunately to download the certs you need to open the port 443 of the letsencrypt container to some random port and query it directly. Haproxy cannot route 443 to one backend port and 80 to another.

You will get a certificate error when you go there because the hostname does not match. Solution could be to open port 80 and 443 for the cert service, bind it to a host machine on 80 and 443 and set the host on which the certificate service is running as a cname under "certs.domain.com". This is what I am doing currently. It wastes a hosts 80 and 443 but there is no other solution due to the limitations of haproxy.

I will have to update the README asap.

[sslash]

Okey. Not so straightforward so I'll wait for the readme docs for this. Maybe a screencast would be ideal? Or screenshots

[levino]

Has been resolved. Closing.