search

ganglia / monitor-core

Ganglia Monitoring core
BSD 3-Clause "New" or "Revised" License
490 stars 246 forks source link

Fix segfault with TCP listener thread on exit #284

Closed junichi-tanaka closed 7 years ago

junichi-tanaka commented 7 years ago

problem

I got segfault below on Ubuntu Xenial:16.04(also Trusty:14.04)

[Sat Apr 22 21:47:52 2017] gmond[29618]: segfault at 7f3d0450f000 ip 00007f3d081195c0 sp 00007ffcdcbf9a00 error 4 in libapr-1.so.0.5.2[7f3d080ff000+31000]
[Sat Apr 22 21:53:25 2017] gmond[29701]: segfault at 7f05ffb42000 ip 00007f060374c5c0 sp 00007ffe68921fb0 error 4 in libapr-1.so.0.5.2[7f0603732000+31000]

backtrace

Thread 1 "gmond" received signal SIGSEGV, Segmentation fault.
0x00007ffff7bbf5c0 in apr_allocator_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0
(gdb) bt
#0  0x00007ffff7bbf5c0 in apr_allocator_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0
#1  0x00007ffff7bc0070 in apr_pool_terminate () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0
#2  0x00007ffff6d4aff8 in __run_exit_handlers (status=0, listp=0x7ffff70d45f8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true) at exit.c:82
#3  0x00007ffff6d4b045 in __GI_exit (status=<optimized out>) at exit.c:104
#4  0x00007ffff6d31837 in __libc_start_main (main=0x404670 <main>, argc=4, argv=0x7fffffffe258, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe248)
    at ../csu/libc-start.c:325
#5  0x00000000004059d9 in _start ()

steps to reproduce

  1. build and install the latest version of monitor-core
  2. generate gmond.conf with "gmond -t"
  3. enable tcp_accept_channel in gmond.conf
  4. deploy python module
  5. start gmond
    • sudo /path/to/gmond -c /path/to/gmond.conf
  6. wait for several seconds
  7. stop gmond
    • sudo kill -TERM `pidof gmond`
  8. confirm segfault
    • dmesg -T | tail -n1

cannot reproduce...