Hot reloading?

[tomodian]

Hi, thank you for making this awesome project!

One question, will this plugin hot load / reload the certs without restarting HAProxy? My project have 10 HAProxy servers which accepts user defined domains (e.g. example.com / myblog.jp / some.other.io) and proxying to backend web servers. For now these traffics are non-TLS and it would be supercool if this plugin could obtain LE Certs and let HAProxy serves TLS connection without restarting.

[kujiy]

I think it's impossible. In my haproxy image(not this project), haproxy works with in-memory certs. It keeps working whenever I delete the certs on the filesystem.

[tomodian]

@kujiy thank you for the information, I found that the plugin have restart feature in this section: https://github.com/greenhost/certbot-haproxy/blob/develop/certbot_haproxy/installer.py#L110

However I assume this only applies to single server which is used to obtain LE Cert and other servers still needs to be restarted manually to reload the certs. Might be out of scope for this plugin but is their any plan to add post hook to ping all running HAProxy servers to propagate restart?

[kujiy]

@tomodian I haven't begun the automatic cert update yet, but I'm afraid that I'll restart all haproxies once in a week or a month by cron with random delay during my working time. Perhaps Monday for server A, Tuesday for server B and so on. We made it a rule that confirming as no problem if the users can see the sites by reloading once or twice. Please teach me if you found a good way!

[SnijderC]

For now these traffics are non-TLS and it would be supercool if this plugin could obtain LE Certs and let HAProxy serves TLS connection without restarting. You can request certificates with this plugin, but you can't load them without restarting due to limitations in HAProxy. However you can 1 of 2 things:

• Start a new HAProxy process with the -sf [pid] argument where pid belongs to the current HAProxy process. This will make the new HAProxy handle new incoming requests while the old process will keep handling requests that belong to the old process, e.g. a file download won't be interrupted and new connections won't be dropped while "restarting". This approach has a disadvantage: you can't do many consecutive restarts in a short period of time, you will eventually have to kill processes, potentially before a large download is completed.
• HAProxy "IP tableflip": http://inside.unbounce.com/product-dev/haproxy-reloads/ which comes down to telling iptables to redirect traffic to a new HAProxy process, CONNTRACK will take care of any existing connections, which will be routed to the old process. If implemented correctly this can be done with zero packet loss. However, if you do not use CONNTRACK, you can't use this "hack".

The second approach seems preferable but we did not yet implement this in a utility script. If you want to work on it, you could fork our project and send us a merge request. BTW we also have a pool of HAProxy servers running so we need a similar solution to what you need.

I found that the plugin have restart feature in this section: https://github.com/greenhost/certbot-haproxy/blob/develop/certbot_haproxy/installer.py#L110

Yes we did implement this but be careful using it because in the current version of certbot, certbot will call this after every successful renewal (issue). If you are renewing 1000 certs, your HAProxy process will be restarted 1000 times within a short time span, which is probably bad news. So currently we use a tiny script that patches up some of the issues described above. Which is not really suitable for open sourcing because it's closely tied to our specific infrastructure, i.e. not generic.

[SnijderC]

Since there was more response for a while, I'm going to close this. Feel free to reopen or send a PR.