colinmollenhour / mariadb-galera-swarm

MariaDb Galera Cluster container based on official mariadb image which can auto-bootstrap and recover cluster state.
https://hub.docker.com/r/colinmollenhour/mariadb-galera-swarm
Apache License 2.0
219 stars 103 forks source link

no-galera-healthcheck.sh in no-galera mode #77

Closed rjsocha closed 5 years ago

rjsocha commented 5 years ago

Hi,

When using no-galera mode with load balancer (haproxy) in front, there are errors logged after each healtcheck.:

2019/06/19 21:23:41 socat[945] E write(8, 0x55c920095b80, 22): Broken pipe

Adding "end-close" to the socat resolved this. For ex:

socat  TCP-LISTEN:8080,crlf,reuseaddr,end-close,fork SYSTEM:"echo HTTP/1.1 200 OK;echo Content-Type\: text/plain;echo;echo \"Fake report: healthy\";"
colinmollenhour commented 5 years ago

Thanks!

rjsocha commented 5 years ago

Unfortunately this solution doesn't work. Sorry After 24h of running there are this errors in log (this is from kontena stack logs):

2019-06-20T18:40:39.587Z [galera.node-1]: 2019/06/20 18:40:39 socat[250] E write(8, 0x5643a9e6ec10, 24): Broken pipe
2019-06-20T18:40:54.959Z [galera.node-1]: 2019/06/20 18:40:54 socat[343] E write(8, 0x5643a9e6ec10, 2): Broken pipe
2019-06-20T18:41:00.843Z [galera.lb-1]:  Server galera-write/node82 is DOWN, reason: Layer7 invalid response, check duration: 52ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2019-06-20T18:41:00.843Z [galera.lb-1]:  backend galera-write has no server available!
2019-06-20T18:41:00.843Z [galera.lb-1]:  Server galera-read/node82 is DOWN via galera-write/node82. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2019-06-20T18:41:00.843Z [galera.lb-1]:  backend galera-read has no server available!
2019-06-20T18:41:00.867Z [galera.node-1]: 2019/06/20 18:41:00 socat[385] E write(7, 0x5643a9e6ac00, 16): Broken pipe
2019-06-20T18:41:01.849Z [galera.lb-1]:  Server galera-write/node82 is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 4ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
2019-06-20T18:41:01.849Z [galera.lb-1]:  Server galera-read/node82 is UP via galera-write/node82. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
2019-06-20T18:41:14.268Z [galera.node-1]: 2019/06/20 18:41:14 socat[463] E write(8, 0x5643a9e6ac00, 22): Broken pipe
2019-06-20T18:41:32.484Z [galera.node-1]: 2019/06/20 18:41:32 socat[591] E write(7, 0x5643a9e6ac00, 16): Broken pipe
2019-06-20T18:43:10.008Z [galera.node-1]: 2019/06/20 18:43:10 socat[1203] E write(7, 0x5643a9e6ac00, 16): Broken pipe
2019-06-20T18:50:38.119Z [galera.node-1]: 2019/06/20 18:50:38 socat[4007] E write(8, 0x5643a9e6ac00, 26): Broken pipe
2019-06-20T18:50:38.122Z [galera.node-1]: sh: echo: I/O error
2019-06-20T18:50:38.122Z [galera.node-1]: sh: echo: I/O error
2019-06-20T18:51:14.316Z [galera.node-1]: 2019/06/20 18:51:14 socat[4236] E write(8, 0x5643a9e6ac00, 22): Broken pipe

For now I'm testing this soultion for no-galera-healthcheck.sh:

#!/bin/sh
[ -d /dev/shm/fakeroot ] || mkdir -p /dev/shm/fakeroot
touch /dev/shm/fakeroot/index.html
while :
do
        echo "Launching Fake Health Check" >&2
        busybox httpd -f -p $1 -h /dev/shm/fakeroot
done

This require additional pacakge in docker image: busybox-static