docker-archive / dockercloud-haproxy

HAproxy image that autoreconfigures itself when used in Docker Cloud
https://cloud.docker.com/
651 stars 187 forks source link

Segmentation Faults #207

Closed markvr closed 7 years ago

markvr commented 7 years ago

hi, I'm getting segmentation faults when running haproxy, especially when an invalid config occurs, most recently when my logging service failed and haproxy couldn't resolve the logging hostname.

I've cross posted this issue to http://discourse.haproxy.org/t/segmentation-fault-when-running-config-check/1304/2 in case the haproxy community has any insight either.

This was discovered when my nodes crashed with no disk space. The haproxy container was 25Gb, and was constantly core dumping 5Mb files. I haven't really been able to isolate the problem entirely, but it doesn't occur if I remove all SSL from my configuration, or when using the official docker "haproxy" image. This version is also more up-to-date than the current one (1.7 vs 1.6.6).

Could this project use a recent release of the official haproxy image?

tifayuki commented 7 years ago

The haproxy container was 25Gb that is weird. Is it because of the 5Mb core dumping files? As to the version of Haproxy, I simply take what the current alpine release gives, which is currently 1.6.6 I think

markvr commented 7 years ago

hi yes it was because of the core dumps. Basically:

I was going to write a pull request to check the config before reloading to avoid breaking a running service, but couldn't even get that far because running haproxy -c -f haproxy.cfg just segfaults each time.

markvr commented 7 years ago

I've just tried testing against this project but with a base image of alpine:3.6 (which equates to haproxy 1.7.5) and it seems to no longer segfault

tifayuki commented 7 years ago

@markvr Haproxy always complains the template error if a name is not resolvable. And I think it is ok to bump the base image version to alpine:3.6. Can you send a PR for that against staging branch?

markvr commented 7 years ago

It will always complain because the config is invalid, but it seems that often (and I've actually also seen this with valid config) it also segfaults. e.g.

/ # haproxy -c -f haproxy.cfg
[WARNING] 162/142118 (8736) : Setting tune.ssl.default-dh-param to 1024 by default, if your workload permits it you should set it to at least 2048. Please set a value >= 1024 to make this warning disappear.
Configuration file is valid
Segmentation fault (core dumped)

I'll send a PR that does a config check before reloading, it will take me a few days to get time to do it.

markvr commented 7 years ago

This is a known issue on v 1.6.6: http://discourse.haproxy.org/t/segmentation-fault-when-running-config-check/1304/3

tifayuki commented 7 years ago

This should be fixed in the new version 1.6.7