pkelsey / libuinet

a library version of FreeBSD's TCP/IP stack plus extras
750 stars 198 forks source link

Segfault in UMA shutdown / destructor path #29

Closed erikarn closed 9 years ago

erikarn commented 9 years ago

After running for a while, a clean shutdown ends with a segfault:

Core was generated by `blockd'. Program terminated with signal 11, Segmentation fault.

0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788

2788 bucket = cache->uc_freebucket; (gdb) bt

0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788

1 0x000000000049485c in zone_free_item (zone=0x802e57000, item=0x8024f5b00, udata=0x0, skip=, flags=0)

at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2932

2 0x0000000000493342 in bucket_drain (zone=0x802e57000, bucket=0x80278f418) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:660

3 local_cache_drain (zone=0x802e57000, cache=) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:670

4 uma_tls_destructor (arg=0x81c00f000) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:1653

5 0x000000080229dd25 in ?? () from /lib/libthr.so.3

6 0x00000008022a533e in ?? () from /lib/libthr.so.3

7 0x00000008022a5289 in ?? () from /lib/libthr.so.3

8 0x00000008022a50fb in pthread_exit () from /lib/libthr.so.3

9 0x000000080229a4fd in ?? () from /lib/libthr.so.3

10 0x0000000000000000 in ?? ()

(gdb) frame 0

0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788

2788 bucket = cache->uc_freebucket; (gdb) print cache $1 = (gdb) print bucket $2 = (gdb) print zone $3 = (uma_zone_t) 0x802e58d80 (gdb) print *zone $4 = {uz_name = 0x4b0ff5 "mbuf_cluster", uz_lock = 0x802e598d0, uz_link = {le_next = 0x0, le_prev = 0x802e59908}, uz_full_bucket = {lh_first = 0x0}, uz_free_bucket = {lh_first = 0x0}, uz_kegs = { lh_first = 0x802e58db8}, uz_klink = {kl_link = {le_next = 0x0, le_prev = 0x802e58db0}, kl_keg = 0x802e598c0}, uz_slab = 0x4963b0 , uz_ctor = 0x432540 , uz_dtor = 0x432620 , uz_init = 0x0, uz_fini = 0x0, uz_flags = 9224, uz_size = 2048, uz_allocs = 245, uz_frees = 0, uz_fails = 0, uz_sleeps = 0, uz_fills = 0, uz_count = 24, uz_cacheidx = 2} (gdb) print uma_tls_key $5 = 3 (gdb) print uhi_tls_get(uma_tls_key) You can't do that without a process to debug. (gdb) frame 4

4 uma_tls_destructor (arg=0x81c00f000) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:1653

1653 local_cache_drain(tls->ut_caches[i].uc_zone, &tls->ut_caches[i]); (gdb)

I'll recompile with -O0 and retry but I bet it's NULL.

erikarn commented 9 years ago

Ok, with -O0:

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 802c09c00 (LWP 100519 nm_rx: netmap0)] 0x000000000050244a in uma_zfree_arg (zone=0x802e59d80, item=0x8008ca800, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788 2788 bucket = cache->uc_freebucket; (gdb) print cache $1 = (uma_cache_t) 0x180 (gdb)

erikarn commented 9 years ago

(gdb) print cache $1 = (uma_cache_t) 0x180 (gdb) print *zone $2 = {uz_name = 0x52a6a3 "mbuf_cluster", uz_lock = 0x802e5a8d0, uz_link = {le_next = 0x0, le_prev = 0x802e5a908}, uz_full_bucket = {lh_first = 0x0}, uz_free_bucket = {lh_first = 0x0}, uz_kegs = {lh_first = 0x802e59db8}, uz_klink = {kl_link = {le_next = 0x0, le_prev = 0x802e59db0}, kl_keg = 0x802e5a8c0}, uz_slab = 0x504bb0 , uz_ctor = 0x4514e0 , uz_dtor = 0x451690 , uz_init = 0x0, uz_fini = 0x0, uz_flags = 9224, uz_size = 2048, uz_allocs = 245, uz_frees = 0, uz_fails = 0, uz_sleeps = 0, uz_fills = 0, uz_count = 24, uz_cacheidx = 2} (gdb) info threads [New Thread 84800f400 (LWP 100527)] [New Thread 802c0b800 (LWP 100526)] [New Thread 802c0b400 (LWP 100525)] [New Thread 802c0b000 (LWP 100524)] [New Thread 802c0ac00 (LWP 100523)] [New Thread 802c0a800 (LWP 100522)] [New Thread 802c0a400 (LWP 100521)] [New Thread 802c0a000 (LWP 100520)] [New Thread 802c09800 (LWP 100518 nm_tx: netmap0)] [New Thread 802c09400 (LWP 100517 one_sighandler)] [New Thread 802c09000 (LWP 100516 shutdown_helper)] [New Thread 802c08400 (LWP 100515 intr: newborn)] [New Thread 802c08000 (LWP 100514 intr: newborn)] [New Thread 802c07c00 (LWP 100513 intr: newborn)] [New Thread 802c07800 (LWP 100512 thread taskq)] [New Thread 802c07400 (LWP 100511 kqueue taskq)] [New Thread 802c07000 (LWP 100510 clock)] [New Thread 802c06c00 (LWP 100509 intr: newborn)] [New Thread 802c06400 (LWP 100139)] Id Target Id Frame 21 Thread 802c06400 (LWP 100139) 0x00000008023268cc in ?? () from /lib/libthr.so.3 20 Thread 802c06c00 (LWP 100509 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3 19 Thread 802c07000 (LWP 100510 clock) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7 18 Thread 802c07400 (LWP 100511 kqueue taskq) 0x00000008023268cc in ?? () from /lib/libthr.so.3 17 Thread 802c07800 (LWP 100512 thread taskq) 0x00000008023268cc in ?? () from /lib/libthr.so.3 16 Thread 802c07c00 (LWP 100513 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3 15 Thread 802c08000 (LWP 100514 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3 14 Thread 802c08400 (LWP 100515 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3 13 Thread 802c09000 (LWP 100516 shutdown_helper) 0x00000008023268cc in ?? () from /lib/libthr.so.3 12 Thread 802c09400 (LWP 100517 one_sighandler) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7 11 Thread 802c09800 (LWP 100518 nm_tx: netmap0) 0x00000008023268cc in ?? () from /lib/libthr.so.3 10 Thread 802c0a000 (LWP 100520) 0x00000008023268cc in ?? () from /lib/libthr.so.3 9 Thread 802c0a400 (LWP 100521) 0x0000000801bdfa9a in _sigwait () from /lib/libc.so.7 8 Thread 802c0a800 (LWP 100522) 0x0000000801c407ea in accept () from /lib/libc.so.7 7 Thread 802c0ac00 (LWP 100523) 0x0000000801cb7bfa in read () from /lib/libc.so.7 6 Thread 802c0b000 (LWP 100524) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7 5 Thread 802c0b400 (LWP 100525) 0x0000000801c9704a in kevent () from /lib/libc.so.7 4 Thread 802c0b800 (LWP 100526) 0x0000000801c407ea in accept () from /lib/libc.so.7 3 Thread 84800f400 (LWP 100527) 0x0000000801cb7bfa in read () from /lib/libc.so.7

pkelsey commented 9 years ago

This is fixed by cdb8e066dd5dee81294c9042363803e31f47832c.