search

codership / glb

Galera Load Balancer - a simple TCP connection proxy and load-balancing library
GNU General Public License v2.0
153 stars 51 forks source link

Crash at startup with discovery feature #10

Open dooblem opened 10 years ago

dooblem commented 10 years ago

Hello,

When starting glbd, it crashes when used with -D flag. What is strange is that I managed to make it work with discovery tomorrow, with now problem. And today it's crashing.

I compiled version 1.0.1 on Ubuntu server 13.10. Simple cluster with 2 Galera nodes.

When adding the 2 nodes directly at startup, it runs OK, but if the cluster membership changes (node remove), it crashes.

I took a very quick look at the code. My guess is there is some parsing problem in glb_dst_parse, but I may be wrong on that.

Thanks in advance, Marc

# /usr/local/sbin/glbd -v --fifo=/var/run/glbd.fifo -D -w exec:"mysql.sh -uroot -pXXXX" --control=127.0.0.1:8011 3306 192.168.56.104:3306
glb v1.0.1 (epoll)
Incoming address: 0.0.0.0:3306, control FIFO: /var/run/glbd.fifo
Control  address:  127.0.0.1:8011
Number of threads: 1, max conn: 493, nodelay: ON, keepalive: ON, defer accept: OFF, linger: OFF, daemon: NO, lat.count: 0, policy: 'least connected', top: NO, verbose: YES
Destinations: 1
   0:  192.168.56.104:3306 , w: 1.000
  DEBUG: glb_wdog.c:156: Adding ' 192.168.56.104:3306 , w: 1.000' at pos. 0
  DEBUG: glb_wdog.c:97: Created context for 192.168.56.104:3306
  DEBUG: glb_wdog_exec.c:194: exec thread: 139983978346240, errno: 0 (Success), pid: 6576, cmd: 'mysql.sh 192.168.56.104:3306 -uroot -ptoto'
  DEBUG: glb_wdog.c:188: Backend thread for '192.168.56.104:3306' started.
  DEBUG: glb_wdog.c:495: main loop collecting...
  DEBUG: glb_wdog.c:495: main loop collecting...
  DEBUG: glb_wdog.c:355: Setting memb_changed because changed_length: 1 or strcmp(
 old: ''
 new: '192.168.56.102:3306,192.168.56.104:3306
'): -49
  DEBUG: glb_wdog.c:568: Changing weight for '192.168.56.104:3306': -1.000 ->  1.000:  0 (Success)
  DEBUG: glb_wdog.c:156: Adding ' 192.168.56.102:3306 , w: 1.000' at pos. 1
  DEBUG: glb_wdog.c:97: Created context for 192.168.56.102:3306
  DEBUG: glb_wdog_exec.c:194: exec thread: 139983961560832, errno: 0 (Success), pid: 6594, cmd: 'mysql.sh 192.168.56.102:3306 -uroot -ptoto'
  DEBUG: glb_wdog.c:188: Backend thread for '192.168.56.102:3306' started.
  ERROR: glb_socket.c:90: Unknown host U.

  ERROR: glb_dst.c:86: Invalid argument
  ERROR: glb_wdog.c:450: Failed to parse destination 'U': 22 (Invalid argument). Skipping.
Watchdog:
------------------------------------------------------------
        Address       : exp  setw     state    lat     curw
 192.168.56.104:3306  :  +   1.000    READY  0.02680   1.000
 192.168.56.102:3306  :      1.000 NOTFOUND  0.00000  -1.000
------------------------------------------------------------
Destinations: 2

Router:
------------------------------------------------------
        Address       :   weight   usage    map  conns
 192.168.56.104:3306  :    1.000   0.000    N/A      0
------------------------------------------------------
Destinations: 1, total connections: 0 of 493 max

Pool: connections per thread:     0

  DEBUG: glb_wdog.c:495: main loop collecting...
  DEBUG: glb_wdog.c:355: Setting memb_changed because changed_length: 0 or strcmp(
 old: '192.168.56.102:3306,192.168.56.104:3306
U'
 new: '192.168.56.102:3306,192.168.56.104:3306
'): 85
  DEBUG: glb_wdog.c:355: Setting memb_changed because changed_length: 1 or strcmp(
 old: '
       $��'
 new: '192.168.56.102:3306,192.168.56.104:3306
'): -37
*** Error in `/usr/local/sbin/glbd': realloc(): invalid pointer: 0x00007f50800008f0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7f4c6)[0x7f508fd5c4c6]
/lib/x86_64-linux-gnu/libc.so.6(realloc+0x300)[0x7f508fd60cf0]
/usr/local/sbin/glbd[0x405273]
/usr/local/sbin/glbd[0x4096ac]
/usr/local/sbin/glbd[0x409f1d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e)[0x7f50900acf6e]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f508fdd79cd]
======= Memory map: ========
00400000-00415000 r-xp 00000000 fc:00 27285                              /usr/local/sbin/glbd
00614000-00615000 r--p 00014000 fc:00 27285                              /usr/local/sbin/glbd
00615000-00616000 rw-p 00015000 fc:00 27285                              /usr/local/sbin/glbd
01b83000-01ba4000 rw-p 00000000 00:00 0                                  [heap]
7f5080000000-7f5080021000 rw-p 00000000 00:00 0 
7f5080021000-7f5084000000 ---p 00000000 00:00 0 
7f5084000000-7f5084022000 rw-p 00000000 00:00 0 
7f5084022000-7f5088000000 ---p 00000000 00:00 0 
7f5088000000-7f5088022000 rw-p 00000000 00:00 0 
7f5088022000-7f508c000000 ---p 00000000 00:00 0 
7f508c08c000-7f508c0a1000 r-xp 00000000 fc:00 153595                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f508c0a1000-7f508c2a0000 ---p 00015000 fc:00 153595                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f508c2a0000-7f508c2a1000 r--p 00014000 fc:00 153595                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f508c2a1000-7f508c2a2000 rw-p 00015000 fc:00 153595                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f508c2a2000-7f508c2a3000 ---p 00000000 00:00 0 
7f508c2a3000-7f508caa3000 rw-p 00000000 00:00 0                          [stack:6611]
7f508caa3000-7f508caa4000 ---p 00000000 00:00 0 
7f508caa4000-7f508d2a4000 rw-p 00000000 00:00 0                          [stack:6610]
7f508d2a4000-7f508d2a6000 r-xp 00000000 fc:00 153290                     /lib/libnss_mdns4.so.2
7f508d2a6000-7f508d4a6000 ---p 00002000 fc:00 153290                     /lib/libnss_mdns4.so.2
7f508d4a6000-7f508d4a7000 r--p 00002000 fc:00 153290                     /lib/libnss_mdns4.so.2
7f508d4a7000-7f508d4a8000 rw-p 00003000 fc:00 153290                     /lib/libnss_mdns4.so.2
7f508d4a8000-7f508d4be000 r-xp 00000000 fc:00 131187                     /lib/x86_64-linux-gnu/libresolv-2.17.so
7f508d4be000-7f508d6be000 ---p 00016000 fc:00 131187                     /lib/x86_64-linux-gnu/libresolv-2.17.so
7f508d6be000-7f508d6bf000 r--p 00016000 fc:00 131187                     /lib/x86_64-linux-gnu/libresolv-2.17.so
7f508d6bf000-7f508d6c0000 rw-p 00017000 fc:00 131187                     /lib/x86_64-linux-gnu/libresolv-2.17.so
7f508d6c0000-7f508d6c2000 rw-p 00000000 00:00 0 
7f508d6c2000-7f508d6c8000 r-xp 00000000 fc:00 131150                     /lib/x86_64-linux-gnu/libnss_dns-2.17.so
7f508d6c8000-7f508d8c7000 ---p 00006000 fc:00 131150                     /lib/x86_64-linux-gnu/libnss_dns-2.17.so
7f508d8c7000-7f508d8c8000 r--p 00005000 fc:00 131150                     /lib/x86_64-linux-gnu/libnss_dns-2.17.so
7f508d8c8000-7f508d8c9000 rw-p 00006000 fc:00 131150                     /lib/x86_64-linux-gnu/libnss_dns-2.17.so
7f508d8c9000-7f508d8cb000 r-xp 00000000 fc:00 153294                     /lib/libnss_mdns4_minimal.so.2
7f508d8cb000-7f508daca000 ---p 00002000 fc:00 153294                     /lib/libnss_mdns4_minimal.so.2
7f508daca000-7f508dacb000 r--p 00001000 fc:00 153294                     /lib/libnss_mdns4_minimal.so.2
7f508dacb000-7f508dacc000 rw-p 00002000 fc:00 153294                     /lib/libnss_mdns4_minimal.so.2
7f508dacc000-7f508dad8000 r-xp 00000000 fc:00 131152                     /lib/x86_64-linux-gnu/libnss_files-2.17.so
7f508dad8000-7f508dcd7000 ---p 0000c000 fc:00 131152                     /lib/x86_64-linux-gnu/libnss_files-2.17.so
7f508dcd7000-7f508dcd8000 r--p 0000b000 fc:00 131152                     /lib/x86_64-linux-gnu/libnss_files-2.17.so
7f508dcd8000-7f508dcd9000 rw-p 0000c000 fc:00 131152                     /lib/x86_64-linux-gnu/libnss_files-2.17.so
7f508dcd9000-7f508dcda000 ---p 00000000 00:00 0 
7f508dcda000-7f508e4da000 rw-p 00000000 00:00 0                          [stack:6593]
7f508e4da000-7f508e4db000 ---p 00000000 00:00 0 
7f508e4db000-7f508ecdb000 rw-p 00000000 00:00 0                          [stack:6577]
7f508ecdb000-7f508ecdc000 ---p 00000000 00:00 0 
7f508ecdc000-7f508f4dc000 rw-p 00000000 00:00 0                          [stack:6575]
7f508f4dc000-7f508f4dd000 ---p 00000000 00:00 0 
7f508f4dd000-7f508fcdd000 rw-p 00000000 00:00 0                          [stack:6574]
7f508fcdd000-7f508fe9a000 r-xp 00000000 fc:00 131095                     /lib/x86_64-linux-gnu/libc-2.17.so
7f508fe9a000-7f509009a000 ---p 001bd000 fc:00 131095                     /lib/x86_64-linux-gnu/libc-2.17.so
7f509009a000-7f509009e000 r--p 001bd000 fc:00 131095                     /lib/x86_64-linux-gnu/libc-2.17.so
7f509009e000-7f50900a0000 rw-p 001c1000 fc:00 131095                     /lib/x86_64-linux-gnu/libc-2.17.so
7f50900a0000-7f50900a5000 rw-p 00000000 00:00 0 
7f50900a5000-7f50900bc000 r-xp 00000000 fc:00 131183                     /lib/x86_64-linux-gnu/libpthread-2.17.so
7f50900bc000-7f50902bc000 ---p 00017000 fc:00 131183                     /lib/x86_64-linux-gnu/libpthread-2.17.so
7f50902bc000-7f50902bd000 r--p 00017000 fc:00 131183                     /lib/x86_64-linux-gnu/libpthread-2.17.so
7f50902bd000-7f50902be000 rw-p 00018000 fc:00 131183                     /lib/x86_64-linux-gnu/libpthread-2.17.so
7f50902be000-7f50902c2000 rw-p 00000000 00:00 0 
7f50902c2000-7f50902e5000 r-xp 00000000 fc:00 131075                     /lib/x86_64-linux-gnu/ld-2.17.so
7f5090458000-7f50904dd000 rw-p 00000000 00:00 0 
7f50904dd000-7f50904e4000 rw-p 00000000 00:00 0 
7f50904e4000-7f50904e5000 r--p 00022000 fc:00 131075                     /lib/x86_64-linux-gnu/ld-2.17.so
7f50904e5000-7f50904e7000 rw-p 00023000 fc:00 131075                     /lib/x86_64-linux-gnu/ld-2.17.so
7fff2cf25000-7fff2cf46000 rw-p 00000000 00:00 0                          [stack]
7fff2cffe000-7fff2d000000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
   INFO: glb_signal.c:42: Received signal 6. Terminating.
Aborted (core dumped)
ayurchen commented 10 years ago

very likely a duplicate of #7