Closed vanbroup closed 6 years ago
I also get a similar issue from time to time. My setup : I have a swarm cluster with several instance of docker-flow-proxy (global on my swarm managers), nib0r/docker-flow-letsencrypt and several services with different hostnames. In front I have a tcp loadbalancer to distribute among my cluster managers. The trigger The trigger could be the deployment of a new service or the reload / restart of the docker daemon or its host. What happens : We receive wrong SSL answer from the proxy. I can get 2 different SSL client errors depending on the case : a generic ERR_SSL_PROTOCOL_ERROR or (more often) a SSL_ERROR_BAD_CERT_DOMAIN (the certificate domain does not match the requested domain). In the latest case, I get a certificate of one of my services for all my services (i.e certificate of my.service1.com when requesting my.service2.com, my.service3.com, ...)
Some times it happens for all services, some other time for part of the services. Some times it happens on all my docker-flow-proxy instances, some other time only the one which has restarted is misconfigured.
I tried with both ssl setup secrets and volumes starting from a clean cluster and got the same issue with both.
I will add seamless-reloads and see if this will solve this issue for you guys.
I found that when I updated the docker-flow-proxy service, some zombie container remained started, side by side with the new one.
These zombie containers was not stopable nor killable using docker stop
/ docker kill
, only a restart would fix it.inge between docker-flow-proxy instances and may have cause the issue (This is just a guess).
If my guess is right, this may be a docker daemon bug (several have been raised about zombie containers. ex: https://github.com/moby/moby/issues/35594).
I upgraded to latest docker version (18.03.0-ce) and I no longer had the issue since then (finger crossed)
That's great news (for DFP, not necessarily for Docker). I'll leave the issue open for a few weeks so that everyone has time to re-test it with newer Docker.
... that does not exclude seamless-reloads. They would be a good addition no matter whether Docker bug caused this in the first place.
The seamless reload feature can be found in dockerflow/docker-flow-proxy:18.05.08-46
or later!
Recreating issue https://github.com/vfarcic/docker-flow-proxy/issues/416
Docker Flow Proxy fails to provision config from time to time (often only on one or more instances), removing DFP from the stack and re-provisioning 'solves' the problem until it reappears again.