search

codership / glb

Galera Load Balancer - a simple TCP connection proxy and load-balancing library
GNU General Public License v2.0
153 stars 51 forks source link

Galera Load Balancer segment fault #8

Open smartdoc opened 10 years ago

smartdoc commented 10 years ago

Hi there!

I was trying to shot GLB for Galera cluster. In normal mode, it works correctly (mostly). During verbose mode, it always take me to segfault.

I test it with CFLAGS -O2, result the same.

I test it on VirtualBox, Debian Wheezy 32-bit, 512Mb ram.

When i start it, i take a segfault:

root@galera1:~# glbd -w exec:"/root/glb-1.0.1/files/mysql.sh -uroot -proot" -t 3 0.0.0.0:3306 galera1:3308 galera2:3308 galera3:3308 glb v1.0.1 (epoll) Incoming address: 0.0.0.0:3306, control FIFO: /tmp/glbd.fifo Control address: none Number of threads: 3, max conn: 493, nodelay: ON, keepalive: ON, defer accept: OFF, linger: OFF, daemon: NO, lat.count: 0, policy: 'least connected', top: NO, verbose: NO Destinations: 3 0: 127.0.1.1:3308 , w: 1.000 1: 192.168.55.114:3308 , w: 1.000 2: 192.168.55.115:3308 , w: 1.000 Watchdog: Address : exp setw state lat curw 127.0.1.1:3308 : + 1.000 READY 0.21315 1.000 192.168.55.114:3308 : + 1.000 READY 0.21868 1.000 192.168.55.115:3308 : + 1.000 NOTFOUND 0.00000 -1.000 Destinations: 3

Router: Address : weight usage map conns 192.168.55.114:3308 : 1.000 0.000 N/A 0 127.0.1.1:3308 : 1.000 0.000 N/A 0 Destinations: 2, total connections: 0 of 493 max

Pool: connections per thread: 0 0 0

* glibc detected * glbd: realloc(): invalid pointer: 0x09b69230 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xb760ff01] /lib/i386-linux-gnu/i686/cmov/libc.so.6(realloc+0x2bb)[0xb7615acb] glbd[0x804d269] ======= Memory map: ======== 08048000-0805d000 r-xp 00000000 08:01 13804 /usr/sbin/glbd 0805d000-0805e000 rw-p 00014000 08:01 13804 /usr/sbin/glbd 09b5b000-09b7c000 rw-p 00000000 00:00 0 [heap] b2ca4000-b2cc0000 r-xp 00000000 08:01 129301 /lib/i386-linux-gnu/libgcc_s.so.1 b2cc0000-b2cc1000 rw-p 0001b000 08:01 129301 /lib/i386-linux-gnu/libgcc_s.so.1 b2cc6000-b2cc7000 ---p 00000000 00:00 0 b2cc7000-b34c7000 rw-p 00000000 00:00 0 b34c7000-b34c8000 ---p 00000000 00:00 0 b34c8000-b3cc8000 rw-p 00000000 00:00 0 b3cc8000-b3cc9000 ---p 00000000 00:00 0 b3cc9000-b44cb000 rw-p 00000000 00:00 0 b44cb000-b44cc000 ---p 00000000 00:00 0 b44cc000-b4ccc000 rw-p 00000000 00:00 0 b4ccc000-b4ccd000 ---p 00000000 00:00 0 b4ccd000-b54cd000 rw-p 00000000 00:00 0 b54cd000-b54ce000 ---p 00000000 00:00 0 b54ce000-b5cce000 rw-p 00000000 00:00 0 b5cce000-b5ccf000 ---p 00000000 00:00 0 b5ccf000-b64cf000 rw-p 00000000 00:00 0 b64cf000-b64d0000 ---p 00000000 00:00 0 b64d0000-b6cd0000 rw-p 00000000 00:00 0 b6cd0000-b6cd1000 ---p 00000000 00:00 0 b6cd1000-b7592000 rw-p 00000000 00:00 0 b7592000-b759c000 r-xp 00000000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b759c000-b759d000 r--p 00009000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b759d000-b759e000 rw-p 0000a000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b759e000-b759f000 rw-p 00000000 00:00 0 b759f000-b76fb000 r-xp 00000000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b76fb000-b76fc000 ---p 0015c000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b76fc000-b76fe000 r--p 0015c000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b76fe000-b76ff000 rw-p 0015e000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b76ff000-b7703000 rw-p 00000000 00:00 0 b7703000-b7718000 r-xp 00000000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so b7718000-b7719000 r--p 00014000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so b7719000-b771a000 rw-p 00015000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so b771a000-b771c000 rw-p 00000000 00:00 0 b771c000-b771e000 r-xp 00000000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so b771e000-b771f000 r--p 00001000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so b771f000-b7720000 rw-p 00002000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so b7720000-b7727000 rw-p 00000000 00:00 0 b7727000-b7728000 r-xp 00000000 00:00 0 [vdso] b7728000-b7744000 r-xp 00000000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so b7744000-b7745000 r--p 0001b000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so b7745000-b7746000 rw-p 0001c000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so bfd7d000-bfd9e000 rw-p 00000000 00:00 0 [stack] INFO: glb_signal.c:42: Received signal 6. Terminating. Aborted

I looked up to messages:

root@galera1:~# cat /var/log/messages Dec 20 15:40:06 galera1 kernel: [ 5426.947387] glbd[4956]: segfault at 135d ip b76452f4 sp b5d2cbdc error 4 in libc-2.13.so[b7601000+15c000

root@galera1:~# glbd -v -w exec:"/root/glb-1.0.1/files/mysql.sh -uroot -proot" -t 3 3306 galera1:3308 galera2:3308 galera3:3308 glb v1.0.1 (epoll) Incoming address: 0.0.0.0:3306, control FIFO: /tmp/glbd.fifo Control address: none Number of threads: 3, max conn: 493, nodelay: ON, keepalive: ON, defer accept: OFF, linger: OFF, daemon: NO, lat.count: 0, policy: 'least connected', top: NO, verbose: YES Destinations: 3 0: 127.0.1.1:3308 , w: 1.000 1: 192.168.55.114:3308 , w: 1.000 2: 192.168.55.115:3308 , w: 1.000 DEBUG: glb_wdog.c:156: Adding ' 127.0.1.1:3308 , w: 1.000' at pos. 0 DEBUG: glb_wdog.c:97: Created context for 127.0.1.1:3308 Segmentation fault

Trying to find in code, testing with debug output, clear out the segfault place: File glb_wdog.c line 186: pthread_create (&ctx->id, NULL, wdog->backend.thread, ctx);

int status; status = pthread_create (&ctx->id, NULL, wdog->backend.thread, ctx); if(status != 0) printf("%i\n", status); // Not Execute !!!

Please, fix it. Take my best wishes. From Russia, with love.