search

ostreedev / ostree

Operating system and container binary deployment and upgrades
https://ostreedev.github.io/ostree/
Other
1.3k stars 296 forks source link

SIGBUS/use-after-free on ARM #381

Closed hadess closed 8 years ago

hadess commented 8 years ago

Following on from #378

During the tests, apart from the test failures, which I'll look at again later on, there are a number of crashers, either SIGBUS, or SIGSEGV ones.

From the 9 core dumps I gathered running the test suite, I found those unique SIGBUS ones:

#0  0xb6c66244 in g_mutex_lock (mutex=mutex@entry=0x32323232) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c34382 in g_source_destroy_internal (source=0x7f8d4d78, context=0x32323232, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c34874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e57042 in soup_session_dispose (object=0x7f5f0898) at soup-session.c:309
#4  0xb6ce49ce in g_object_unref (_object=0x7f5f0898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ef2282 in thread_closure_unref (thread_closure=0x7f601c18) at src/libostree/ostree-fetcher.c:143
#6  0xb6ef23e6 in ostree_fetcher_session_thread (data=0x7f601c18) at src/libostree/ostree-fetcher.c:486
#7  0xb6c50ffa in g_thread_proxy (data=0x7f5f5d20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#8  0xb6f1bf88 in start_thread (arg=0xb62cb360) at pthread_create.c:311
#9  0xb6b261fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6c78244 in g_mutex_lock (mutex=mutex@entry=0xb9b9b9b9) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c46382 in g_source_destroy_internal (source=0x7f789b00, context=0xb9b9b9b9, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c46874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e69042 in soup_session_dispose (object=0x7f5d5898) at soup-session.c:309
#4  0xb6cf69ce in g_object_unref (_object=0x7f5d5898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6f04282 in thread_closure_unref (thread_closure=0x7f5c6c18) at src/libostree/ostree-fetcher.c:143
#6  0xb6f04eda in _ostree_fetcher_finalize (object=0x7f5cb3b0) at src/libostree/ostree-fetcher.c:537
#7  0xb6cf6a48 in g_object_unref (_object=0x7f5cb3b0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#8  0xb6f05dd0 in _ostree_metalink_finalize (object=0x7f5cf6a0) at src/libostree/ostree-metalink.c:378
#9  0xb6cf6a48 in g_object_unref (_object=0x7f5cf6a0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#10 0xb6eed54a in glnx_local_obj_unref (v=<synthetic pointer>) at ./libglnx/glnx-local-alloc.h:63
#11 ostree_repo_pull_with_options (self=self@entry=0x7f5c4078, remote_name_or_baseurl=<optimized out>, options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbeca5fd0) at src/libostree/ostree-repo-pull.c:2176
#12 0x7f59da02 in ostree_builtin_pull (argc=2, argv=0xbeca6174, cancellable=<optimized out>, error=0xbeca5fd0) at src/ostree/ot-builtin-pull.c:260
#13 0x7f5988ec in ostree_run (argc=3, argv=0xbeca6174, commands=0x7f5b8008 <commands>, res_error=0xbeca6000) at src/ostree/ot-main.c:209
#14 0x7f593166 in main (argc=4, argv=0xbeca6174) at src/ostree/main.c:78

And a couple of SEGVs too:

#0  0xb6c9f240 in g_mutex_lock (mutex=mutex@entry=0x3747470) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6c6d382 in g_source_destroy_internal (source=0x7f74e778, context=0x3747470, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6c6d874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e90042 in soup_session_dispose (object=0x7f597098) at soup-session.c:309
#4  0xb6d1d9ce in g_object_unref (_object=0x7f597098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6f2b282 in thread_closure_unref (thread_closure=0x7f5a9018) at src/libostree/ostree-fetcher.c:143
#6  0xb6f2b3e6 in ostree_fetcher_session_thread (data=0x7f5a9018) at src/libostree/ostree-fetcher.c:486
#7  0xb6c89ffa in g_thread_proxy (data=0x7f59df20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#8  0xb6f54f88 in start_thread (arg=0xb6304360) at pthread_create.c:311
#9  0xb6b5f1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6bdd1da in g_source_unref_internal (source=0x7f6b2898, context=0x0, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2012
#1  0xb6bde296 in g_source_unref (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2063
#2  0xb6e00048 in soup_session_dispose (object=0x7f5c1098) at soup-session.c:310
#3  0xb6c8d9ce in g_object_unref (_object=0x7f5c1098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#4  0xb6e9b282 in thread_closure_unref (thread_closure=0x7f5d2418) at src/libostree/ostree-fetcher.c:143
#5  0xb6e9b3e6 in ostree_fetcher_session_thread (data=0x7f5d2418) at src/libostree/ostree-fetcher.c:486
#6  0xb6bf9ffa in g_thread_proxy (data=0x7f5c8120) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6ec4f88 in start_thread (arg=0xb6279360) at pthread_create.c:311
#8  0xb6acf1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

#0  0xb6f2d44c in pending_uri_unref (pending=0x0) at src/libostree/ostree-fetcher.c:208
208   if (!g_atomic_int_dec_and_test (&pending->ref_count))
(gdb) bt
#0  0xb6f2d44c in pending_uri_unref (pending=0x0) at src/libostree/ostree-fetcher.c:208
#1  0xb6c6484a in g_hash_table_remove_all_nodes (hash_table=0x7f63ed88, notify=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/ghash.c:503
#2  0xb6c64ede in g_hash_table_unref (hash_table=0x7f63ed88) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/ghash.c:1042
#3  0xb6f2d2d8 in thread_closure_unref (thread_closure=0x7f64e870) at src/libostree/ostree-fetcher.c:162
#4  0xb6f2deda in _ostree_fetcher_finalize (object=0x7f645398) at src/libostree/ostree-fetcher.c:537
#5  0xb6d1fa48 in g_object_unref (_object=0x7f645398) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#6  0xb6f1687a in ostree_repo_pull_with_options (self=self@entry=0x7f64c078, remote_name_or_baseurl=<optimized out>, options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbeeb6fd0) at src/libostree/ostree-repo-pull.c:2853
#7  0x7f617a02 in ostree_builtin_pull (argc=3, argv=0xbeeb7174, cancellable=<optimized out>, error=0xbeeb6fd0) at src/ostree/ot-builtin-pull.c:260
#8  0x7f6128ec in ostree_run (argc=4, argv=0xbeeb7174, commands=0x7f632008 <commands>, res_error=0xbeeb7000) at src/ostree/ot-main.c:209
#9  0x7f60d166 in main (argc=5, argv=0xbeeb7174) at src/ostree/main.c:78

#0  0xb6c1c240 in g_mutex_lock (mutex=mutex@entry=0x50505050) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6bea382 in g_source_destroy_internal (source=0x7f75adc0, context=0x50505050, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6bea874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6e0d042 in soup_session_dispose (object=0x7f5b3898) at soup-session.c:309
#4  0xb6c9a9ce in g_object_unref (_object=0x7f5b3898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ea8282 in thread_closure_unref (thread_closure=0x7f5c3008) at src/libostree/ostree-fetcher.c:143
#6  0xb6ea8eda in _ostree_fetcher_finalize (object=0x7f5b9338) at src/libostree/ostree-fetcher.c:537
#7  0xb6c9aa48 in g_object_unref (_object=0x7f5b9338) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3170
#8  0xb6e923aa in ostree_repo_pull_with_options (self=self@entry=0x7f5c0078, remote_name_or_baseurl=0x7f5bfe58 "origin", options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbebb3fc0) at src/libostree/ostree-repo-pull.c:2653
#9  0x7f58ba02 in ostree_builtin_pull (argc=2, argv=0xbebb4164, cancellable=<optimized out>, error=0xbebb3fc0) at src/ostree/ot-builtin-pull.c:260
#10 0x7f5868ec in ostree_run (argc=5, argv=0xbebb4164, commands=0x7f5a6008 <commands>, res_error=0xbebb3ff0) at src/ostree/ot-main.c:209
#11 0x7f581166 in main (argc=6, argv=0xbebb4164) at src/ostree/main.c:78

The packages are:

ii  libsoup2.4-1:armhf                     2.48.0-1                                                 armhf        HTTP library implementation in C -- Shared library
ii  libglib2.0-0:armhf                     2.42.1-1+b1                                              armhf        GLib library of C routines
ii  ostree                                 2016.6-2.1                                               armhf        content-addressed filesystem for operating system binaries

They all seem related to a refcounting/access after free issue. libglnx bug?

cgwalters commented 8 years ago

I think this is a dup of #373

cgwalters commented 8 years ago

@hadess can you try the patches in https://github.com/ostreedev/ostree/pull/383 too?

hadess commented 8 years ago

Do I need both the patches in the #383 PR and the libsoup changes?

cgwalters commented 8 years ago

Try with just #383 first.

hadess commented 8 years ago

After running the whole test suite, I get similar traces to this times 5 (either SIGBUS or SIGSEGV):

#0  0xb6be2244 in g_mutex_lock (mutex=mutex@entry=0x11111111) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread-posix.c:1337
#1  0xb6bb0382 in g_source_destroy_internal (source=0x13c6d0, context=0x11111111, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1159
#2  0xb6bb0874 in g_source_destroy (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:1227
#3  0xb6dd3042 in soup_session_dispose (object=0x4a098) at soup-session.c:309
#4  0xb6c609ce in g_object_unref (_object=0x4a098) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6e6c00e in ostree_fetcher_session_thread (data=0x5a818) at src/libostree/ostree-fetcher.c:486
#6  0xb6bccffa in g_thread_proxy (data=0x50d20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6e94f88 in start_thread (arg=0xb627c360) at pthread_create.c:311
#8  0xb6aa21fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
hadess commented 8 years ago

That was with ostree git master.

cgwalters commented 8 years ago

Interesting. The patches from #383 definitely made things more reliable for me (x86_64/CentOS7+ CAHC).

hadess commented 8 years ago

So, libsoup gnome-3-14 + patch from GNOME bz, and the ostree master, and it's not working any better from what I can tell.

Thread 3 (Thread 0xb591d360 (LWP 27436)):
#0  0xb6ad79d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bee39e in g_main_context_poll (priority=2147483647, n_fds=2, fds=0x288a60, timeout=60000, context=0x302a00) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=0x302a00, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bee668 in g_main_loop_run (loop=0x302e48) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3975
#4  0xb6ea7fdc in ostree_fetcher_session_thread (data=0x5cc70) at src/libostree/ostree-fetcher.c:478
#5  0xb6c08ffa in g_thread_proxy (data=0x50f50) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#6  0xb6ed0f88 in start_thread (arg=0xb591d360) at pthread_create.c:311
#7  0xb6ade1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb6f93000 (LWP 27434)):
#0  0xb6ad79d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bee39e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x682f8, timeout=-1, context=0x1ef400) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x1ef400, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bee44c in g_main_context_iteration (context=context@entry=0x1ef400, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6ea9282 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x502c0, uri=uri@entry=0x56000, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=out_contents@entry=0xbeedddf0, 
    max_size=max_size@entry=10485760, cancellable=0x0, error=error@entry=0xbeede024) at src/libostree/ostree-fetcher.c:1184
#5  0xb6e93158 in fetch_uri_contents_membuf_sync (pull_data=<optimized out>, pull_data=<optimized out>, error=0xbeede024, cancellable=<optimized out>, out_contents=0xbeedddf0, allow_noent=1, add_nul=0, 
    uri=0x56000) at src/libostree/ostree-repo-pull.c:374
#6  request_static_delta_superblock_sync (error=0xbeede024, cancellable=<optimized out>, out_delta_superblock=<synthetic pointer>, 
    to_revision=0x175930 "b7f70a14a776b7bd3fc2d78fee80a8321ecceb0170c292ee8de390e69d5cf030", from_revision=0x0, pull_data=0xbeedde30) at src/libostree/ostree-repo-pull.c:1436
#7  ostree_repo_pull_with_options (self=self@entry=0x59078, remote_name_or_baseurl=remote_name_or_baseurl@entry=0x58c68 "R1", options=<optimized out>, progress=progress@entry=0x0, 
    cancellable=cancellable@entry=0x0, error=error@entry=0xbeede024) at src/libostree/ostree-repo-pull.c:2710
#8  0x000222c0 in ostree_builtin_pull (argc=2, argv=0xbeede1b4, cancellable=0x0, error=0xbeede024) at src/ostree/ot-builtin-pull.c:260
#9  0x0001d8ec in ostree_run (argc=3, argc@entry=4, argv=argv@entry=0xbeede1b4, commands=commands@entry=0x3cc80 <commands>, res_error=res_error@entry=0xbeede04c) at src/ostree/ot-main.c:209
#10 0x000189ee in main (argc=4, argv=0xbeede1b4) at src/ostree/main.c:78

Thread 1 (Thread 0xb62eb360 (LWP 27435)):
#0  0x0041e348 in ?? ()
#1  0xb6bec1e2 in g_source_unref_internal (source=0x1ef4f8, context=0x0, have_lock=have_lock@entry=0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2016
#2  0xb6bed296 in g_source_unref (source=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:2063
#3  0xb6e0f048 in soup_session_dispose (object=0x4b898) at soup-session.c:310
#4  0xb6c9c9ce in g_object_unref (_object=0x4b898) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./gobject/gobject.c:3133
#5  0xb6ea800e in ostree_fetcher_session_thread (data=0x5cc18) at src/libostree/ostree-fetcher.c:486
#6  0xb6c08ffa in g_thread_proxy (data=0x50f20) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#7  0xb6ed0f88 in start_thread (arg=0xb62eb360) at pthread_create.c:311
#8  0xb6ade1fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
hadess commented 8 years ago

Sorry, I didn't install libsoup properly (the /usr/lib/ files are namespaced as well). So, with libsoup from gnome-3-14 with the patch added on top.

It hangs much sooner in the test suite, when running /home/chip/ostree/tests/.libs/lt-test-pull-c -k --tap:

#0  0xb690f9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb69a939e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x557a0, timeout=-1, context=0x229038) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x229038, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb69a944c in g_main_context_iteration (context=context@entry=0x229038, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6f9e2a2 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x32830, uri=uri@entry=0x37f60, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=0xbebd3d98, max_size=10485760, 
    cancellable=0x0, error=0xbebd3fb4) at src/libostree/ostree-fetcher.c:1184
#5  0xb6f88a5c in fetch_uri_contents_membuf_sync (pull_data=0xbebd3dd8, pull_data=0xbebd3dd8, error=0xbebd3fb4, cancellable=0x0, out_contents=0xbebd3d98, allow_noent=1, add_nul=0, uri=0x37f60)
    at src/libostree/ostree-repo-pull.c:374
#6  ostree_repo_pull_with_options (self=self@entry=0x39078, remote_name_or_baseurl=remote_name_or_baseurl@entry=0x114f0 "origin", options=options@entry=0x328c0, progress=progress@entry=0x0, 
    cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo-pull.c:2431
#7  0xb6f7fb5a in ostree_repo_pull_one_dir (self=0x39078, remote_name=remote_name@entry=0x114f0 "origin", dir_to_pull=dir_to_pull@entry=0x0, refs_to_fetch=refs_to_fetch@entry=0xbebd3fb8, 
    flags=flags@entry=OSTREE_REPO_PULL_FLAGS_NONE, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo.c:3791
#8  0xb6f7fba0 in ostree_repo_pull (self=<optimized out>, remote_name=remote_name@entry=0x114f0 "origin", refs_to_fetch=refs_to_fetch@entry=0xbebd3fb8, flags=flags@entry=OSTREE_REPO_PULL_FLAGS_NONE, 
    progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, error=error@entry=0xbebd3fb4) at src/libostree/ostree-repo.c:3749
#9  0x00011120 in test_pull_multi_nochange (data=0xbebd414c) at tests/test-pull-c.c:77
#10 0xb69c315e in test_case_run (tc=0x32a18) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2059
#11 g_test_run_suite_internal (suite=suite@entry=0x4d930, path=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2120
#12 0xb69c327c in g_test_run_suite_internal (suite=suite@entry=0x4d940, path=<optimized out>, path@entry=0xb6a12e08 "") at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2131
#13 0xb69c3508 in g_test_run_suite (suite=0x4d940) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:2184
#14 0xb69c3556 in g_test_run () at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gtestutils.c:1488
#15 0x00010f68 in main (argc=1, argv=0xbebd42c4) at tests/test-pull-c.c:129

And just to make sure:

$ cat /proc/7333/smaps  | grep libsoup
b6d7f000-b6de7000 r-xp 00000000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
b6de7000-b6df6000 ---p 00068000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
b6df6000-b6e05000 rw-p 00067000 00:0f 428638     /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
$ ls -l /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
-rwxr-xr-x 1 root root 1907800 Jul 11 19:19 /usr/lib/arm-linux-gnueabihf/libsoup-2.4.so.1.7.0
$ date
Mon 11 Jul 19:39:12 UTC 2016

That really was the library compiled before I ran the test suite.

I'll try running against libsoup master + that patch now, if the dependencies allow me.

hadess commented 8 years ago

OK, libsoup master + the patch gets stuck much further, running test-remote-gpg-import.sh I think:

chip     21496  0.0  2.0  27300 10300 pts/0    Sl+  20:02   0:00 ostree --repo=repo pull R1:main
Thread 2 (Thread 0xb6235360 (LWP 21497)):
#0  0xb6abc9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bd339e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x7f759748, timeout=-1, context=0x7f6504f8) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=0x7f6504f8, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bd3668 in g_main_loop_run (loop=0x7f650ed8) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3975
#4  0xb6e94fdc in ostree_fetcher_session_thread (data=0x7f63e418) at src/libostree/ostree-fetcher.c:478
#5  0xb6bedffa in g_thread_proxy (data=0x7f648120) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gthread.c:764
#6  0xb6ebdf88 in start_thread (arg=0xb6235360) at pthread_create.c:311
#7  0xb6ac31fc in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb6f18000 (LWP 21496)):
#0  0xb6abc9d0 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0xb6bd339e in g_main_context_poll (priority=2147483647, n_fds=1, fds=0x7f7eabf0, timeout=-1, context=0x7f651fa0) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:4076
#2  g_main_context_iterate (context=context@entry=0x7f651fa0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3776
#3  0xb6bd344c in g_main_context_iteration (context=context@entry=0x7f651fa0, may_block=may_block@entry=1) at /build/glib2.0-5Ukv2U/glib2.0-2.42.1/./glib/gmain.c:3842
#4  0xb6e96282 in _ostree_fetcher_request_uri_to_membuf (fetcher=0x7f650960, uri=uri@entry=0x7f652b20, add_nul=add_nul@entry=0, allow_noent=allow_noent@entry=1, out_contents=0xbea8dde0, max_size=10485760, 
    cancellable=0x0, error=0xbea8e020) at src/libostree/ostree-fetcher.c:1184
#5  0xb6e80a3c in fetch_uri_contents_membuf_sync (pull_data=0xbea8de20, pull_data=0xbea8de20, error=0xbea8e020, cancellable=0x0, out_contents=0xbea8dde0, allow_noent=1, add_nul=0, uri=0x7f652b20)
    at src/libostree/ostree-repo-pull.c:374
#6  ostree_repo_pull_with_options (self=self@entry=0x7f64e078, remote_name_or_baseurl=0x7f64f788 "R1", options=<optimized out>, progress=progress@entry=0x0, cancellable=cancellable@entry=0x0, 
    error=error@entry=0xbea8e020) at src/libostree/ostree-repo-pull.c:2431
#7  0x7f6199e2 in ostree_builtin_pull (argc=2, argv=0xbea8e1c4, cancellable=<optimized out>, error=0xbea8e020) at src/ostree/ot-builtin-pull.c:260
#8  0x7f6148cc in ostree_run (argc=3, argv=0xbea8e1c4, commands=0x7f634008 <commands>, res_error=0xbea8e050) at src/ostree/ot-main.c:209
#9  0x7f60f146 in main (argc=4, argv=0xbea8e1c4) at src/ostree/main.c:78

But the SIGBUS errors in test-basic.sh are gone. I'll try to bisect, so we can choose a good minimal requirement, or backport the fixes.

hadess commented 8 years ago

It crashes without the libsoup patch on top of master, and I still see random "hangs" as above, with the application being stuck waiting with the patch applied (also on top of master).