search

GrapheneOS / hardened_malloc

Hardened allocator designed for modern systems. It has integration into Android's Bionic libc and can be used externally with musl and glibc as a dynamic library for use on other Linux-based platforms. It will gain more portability / integration over time.
https://grapheneos.org/
MIT License
1.27k stars 97 forks source link

sssd crashes with hardened_malloc in ld preload #98

Open mike2307 opened 5 years ago

mike2307 commented 5 years ago

sssd on fedora 31 reports a segementation fault when running with lib hardened malloc.

Package version: sssd-common-2.2.2-1.fc31 Crashing function: ldb_kv_dn_list_load

Stack trace:

Process 5945 (sssd) of user 0 dumped core.
Stack trace of thread 5945: #0 0x00007f50329388b0 _int_free (libc.so.6)
#1 0x000072d6d7852f0d ldb_kv_dn_list_load (libldb-key-value.so)
#2 0x000072d6d7854f24 ldb_kv_index_add1 (libldb-key-value.so)
#3 0x000072d6d7856cce ldb_kv_index_add_all (libldb-key-value.so)
#4 0x000072d6d785714e ldb_kv_index_add_new (libldb-key-value.so)
#5 0x000072d6d784fcb9 ldb_kv_add_internal (libldb-key-value.so)
#6 0x000072d6d7850a58 ldb_kv_callback (libldb-key-value.so)
#7 0x00007f5032a96f3d tevent_common_invoke_timer_handler (libtevent.so.0)
#8 0x00007f5032a970e0 tevent_common_loop_timer_delay (libtevent.so.0)
#9 0x00007f5032a981bc epoll_event_loop_once (libtevent.so.0)
#10 0x00007f5032a9653b std_event_loop_once (libtevent.so.0)
#11 0x00007f5032a915d8 _tevent_loop_once (libtevent.so.0)
#12 0x00007f5032bed47b ldb_wait (libldb.so.2)
#13 0x00007f5032bee033 ldb_autotransaction_request (libldb.so.2)
#14 0x00007f5032beee45 ldb_add (libldb.so.2)
#15 0x0000558a4ad704c1 confdb_setup (sssd)
#16 0x0000558a4ad6dc11 load_configuration (sssd)
#17 0x0000558a4ad69f33 main (sssd)
#18 0x00007f50328d61a3 __libc_start_main (libc.so.6)
#19 0x0000558a4ad6a80e _start (sssd)
mike2307 commented 5 years ago

The corresponding bugzilla ticket is https://bugzilla.redhat.com/show_bug.cgi?id=1761232

thestinger commented 5 years ago

Most issues like this are upstream memory corruption bugs uncovered by hardened_malloc. It uncovers latent memory corruption bugs as part of mitigating exploitation. However, in this case, I see _int_free in the backtrace which indicates that glibc malloc is being at least partially used. I think something unusual is happening here. The malloc implementation isn't being properly substituted via the mechanism you're using, perhaps due to something odd about how the software is compiled / linked. Even though glibc officially supports using linker preloading for malloc replacement, there are edge cases that aren't supported properly by them. I'm curious if you encounter a crash using jemalloc or another malloc implementation via the same replacement mechanism that you're using.

mike2307 commented 5 years ago

When using jemalloc, sssd works. I'm getting some SELinux errors regarding unix_chkpwd instead...

mike2307 commented 5 years ago

In the RPM spec file of the failing library, I see a dependency to library called 'talloc'. See libldb.spec.txt. Maybe that's related somehow...

mike2307 commented 5 years ago

By the way; This is the method I'm using to replace the malloc/free calls:

# cat /etc/ld.so.preload
/usr/local/lib64/libhardened_malloc.so

The library is compiled with all settings to their defaults, using the latest version on master.

mike2307 commented 5 years ago

I ran the application with

LD_PRELOAD=/usr/local/lib64/libhardened_malloc.so LD_DEBUG=all sssd 2> hardened.txt
LD_PRELOAD=/usr/local/lib64/libjemalloc.so LD_DEBUG=all sssd 2> jemalloc.txt

The output is attached sssdtest.tar.gz. The interesting part starts at hardened.txt, line 134592. From that point onwards, libhardened_malloc is not considered any more when looking up symbols. The matching section in jemalloc.txt start at line 138061. libjemalloc however is used for resolving symbols just fine afterwards...

Edit: Seems like i misaligned the two files. Actually the look quite similar. But one works; the other not. Maybe the debug output contains some hints elsewhere...

thestinger commented 4 years ago

@mike2307 Is this still an issue?

mike2307 commented 4 years ago

@thestinger Yes, this issue is still valid.

thestinger commented 3 years ago

@mike2307 Can you try with v6 of hardened_malloc? There have been various little fixes over time since I last asked.

mike2307 commented 3 years ago

@thestinger: No, sorry. The issue is still present with the latest version.