Open wowu opened 9 months ago
https://doc.traefik.io/traefik/middlewares/http/retry/ maybe this can help ?
I like the current idea, but wouldn't the "sleep" timout need to be at least the interval of the health check times the "retry" amount for the health check? Otherwise the container might not enter the "unhealthy" state before the shutdown sequence starts.
Maybe another option (specifically for traefik): in the --before-stop
hook, add the label traefik.enable=false
. (note that I have not tested how quickly traefik would pick up this change).
Currently there's no way of telling Traefik that old container is going to be stopped, so it might route requests to a container that is shutting down. I'm creating this issue to track the progress of figuring out what is the best way of implementing this.
The problem was mentioned in this StackOverflow question: https://stackoverflow.com/questions/75918681/how-to-avoid-downtime-when-using-docker-rollout-with-traefik
Current idea
The easiest way seems to fail healthchecks before the container is going to be stopped, so Traefik is not routing new requests to the unhealthy container(s). This can be achieved by adding
! test -f /drain
to container healthcheck, that is "fail if there exists a file nameddrain
in/
", anddocker-rollout
can create this file before stopping the old container.I'm not sure if this behavior should be hardcoded in the tool, as there might be better ways of implementing request draining for proxies other than Traefik / nginx. Implementing hook support would allow
docker-rollout
users to implement true zero downtime deployment in two steps:&& ! test -f /drain
to current container healtcheck in compose file--before-stop "docker exec $1 touch /drain && sleep 10"
to create the file manually