search

smebberson / docker-alpine

Docker containers running Alpine Linux and s6 for process management. Solid, reliable containers.
MIT License
596 stars 186 forks source link

consul-template stopped after consul leads to errors #48

Closed matthewvalimaki closed 8 years ago

matthewvalimaki commented 8 years ago

If consul-template in alpine-consul-base is started the process will be killed after consul has already gone away leading to errors such as

[ERR] (view) "key(ssl/crt)" store key: error fetching: Get http://127.0.0.1:8500/v1/kv/ssl/crt?stale=&wait=60000ms: dial tcp 127.0.0.1:8500: getsockopt: connection refused

Also it seems that s6-svc -h /var/run/s6/services/consul-template does not kill the old process. Actually s6-svc -k has no effect at all. I couldn't kill the process no matter what flag I used.

If I manually kill the consul-template process then error above goes away. There is consul-template -pid-file which I tried having and then using /etc/cont-finish.d/#consul-template (weird name to have it execute before 00-consul) with kill -9cat /var/run/consul-template.pid`` but apparently it doesn't die quickly enough as sometimes the error above would still show up.

Any ideas?

smebberson commented 8 years ago

@matthewvalimaki I presume you're talking about the new development branch that we're working on https://github.com/smebberson/docker-alpine/tree/alpine-consul-ui-upgrades.

Anyway, I tested in such and have just pushed a couple of minor changes which seems to have cleared these issues up.

I also found that s6-svc -h /var/run/s6/services/consul-template does NOT kill the old process. However, with the recent changes you will notice output stating Received HUP, reloading configuration... so at least the signals are getting through now.

Using s6-svc -k /var/run/s6/services/consul-template no longer leaves zombie processes lying around. The old one disappears and a new one is made in its place.

That seems to have solved it?! Did you wanna confirm for me?

matthewvalimaki commented 8 years ago

@smebberson I was testing against latest release assuming that it'd carry over as consul-template specific scripts did not change. I think it's a bug in consul-template and its daemon setup. I have not yet taken a stab at it but it irritates me quite a bit.

I'll test your changes tonight. Does s6-svc -k /var/run/s6/services/consul-template && s6-svc -d /var/run/s6/services/consul-template keep it down?

matthewvalimaki commented 8 years ago

@smebberson looks good! I'm not happy with consul-template but that's another story.

smebberson commented 8 years ago

Closing this as everything is working in the latest releases.