drcachesim function tracing uses malloc which is disallowed for static linkage

[derekbruening]

drcachesim wants to support embedded static linkage use, and thus avoids malloc at runtime. When -record_heap is used it violates that:

$ bin64/drrun -t drcachesim -simulator_type func_view -record_heap -offline -- suite/tests/bin/simple_app

<Application /home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/simple_app (71659) DynamoRIO usage error : malloc invoked mid-run when disallowed by DR_DISALLOW_UNSAFE_STATIC>

#0  report_dynamorio_problem (dcontext=0x0, dumpcore_flag=16, exception_addr=0x0, report_ebp=0x0, fmt=0x7ffff7eb6f57 "Usage error: %s (%s, line %d)")
    at /home/bruening/dr/git/src/core/utils.c:2140
#1  0x00007ffff7c5c033 in external_error (file=0x7ffff7f14da8 "/home/bruening/dr/git/src/core/unix/module.c", line=685, 
    msg=0x7ffff7f15320 "malloc invoked mid-run when disallowed by DR_DISALLOW_UNSAFE_STATIC") at /home/bruening/dr/git/src/core/utils.c:201
#2  0x00007ffff7e6941d in redirect_malloc_initonly (size=136) at /home/bruening/dr/git/src/core/unix/module.c:685
#3  0x00007fffb3c3c49e in _libelf_allocate_elf () at libelf_allocate.c:46
#4  0x00007fffb3c3e098 in _libelf_memory (image=0x7ffff70e2000 "\177ELF\002\001\001\003", sz=1820104, reporterror=1) at libelf_memory.c:52
#5  0x00007fffb3c1bbc7 in drsym_obj_mod_init_pre (map_base=0x7ffff70e2000 "\177ELF\002\001\001\003", map_size=1820104)
    at /home/bruening/dr/git/src/ext/drsyms/drsyms_elf.c:225
#6  0x00007fffb3c15a17 in load_module (modpath=0x7fff33c3e370 "/lib/x86_64-linux-gnu/libc-2.29.so")
    at /home/bruening/dr/git/src/ext/drsyms/drsyms_unix_common.c:140
#7  0x00007fffb3c1650d in drsym_unix_load (modpath=0x7fff33c3e370 "/lib/x86_64-linux-gnu/libc-2.29.so")
    at /home/bruening/dr/git/src/ext/drsyms/drsyms_unix_common.c:449
#8  0x00007fffb3c150eb in lookup_or_load (modpath=0x7fff33c3e370 "/lib/x86_64-linux-gnu/libc-2.29.so")
    at /home/bruening/dr/git/src/ext/drsyms/drsyms_unix_frontend.c:76
#9  0x00007fffb3c15227 in drsym_lookup_symbol_local (modpath=0x7fff33c3e370 "/lib/x86_64-linux-gnu/libc-2.29.so", symbol=0x7fffb3bc50e8 "tc_malloc", 
    modoffs=0x7fff33c02708, flags=1) at /home/bruening/dr/git/src/ext/drsyms/drsyms_unix_frontend.c:121
#10 0x00007fffb3c1551d in drsym_lookup_symbol (modpath=0x7fff33c3e370 "/lib/x86_64-linux-gnu/libc-2.29.so", symbol=0x7fffb3bc50e8 "tc_malloc", 
    modoffs=0x7fff33c02708, flags=1) at /home/bruening/dr/git/src/ext/drsyms/drsyms_unix_frontend.c:256
#11 0x00007fffb3b9d417 in get_pc_by_symbol (mod=0x7fff33bc3ce8, symbol=0x7fffb3bc50e8 "tc_malloc")
    at /home/bruening/dr/git/src/clients/drcachesim/tracer/func_trace.cpp:161
#12 0x00007fffb3b9d5ba in instru_funcs_module_load (drcontext=0x7fff33bb3e40, mod=0x7fff33bc3ce8, loaded=true)
    at /home/bruening/dr/git/src/clients/drcachesim/tracer/func_trace.cpp:191
#13 0x00007fffb3c74289 in drmgr_modload_event (drcontext=0x7fff33bb3e40, info=0x7fff33bc3ce8, loaded=1 '\001')
    at /home/bruening/dr/git/src/ext/drmgr/drmgr.c:1666
#14 0x00007ffff7d64dd1 in instrument_module_load (data=0x7fff33bc3ce8, previously_loaded=true) at /home/bruening/dr/git/src/core/lib/instrument.c:2074
#15 0x00007ffff7d64cde in instrument_module_load_trigger (pc=0x7ffff732a490 "H\213\025\311\331\022") at /home/bruening/dr/git/src/core/lib/instrument.c:2048
#16 0x00007ffff7d1b720 in check_thread_vm_area (dcontext=0x7fff33bb3e40, pc=0x7ffff732a490 "H\213\025\311\331\022", tag=0x7ffff732a490 "H\213\025\311\331\022", 
    vmlist=0x7fff33c02db0, flags=0x7fff33c02da8, stop=0x7fff33c02df8, xfer=false) at /home/bruening/dr/git/src/core/vmareas.c:7466
#17 0x00007ffff7dda87b in check_new_page_start (dcontext=0x7fff33bb3e40, bb=0x7fff33c02d70) at /home/bruening/dr/git/src/core/arch/interp.c:720
#18 0x00007ffff7deaee2 in init_interp_build_bb (dcontext=0x7fff33bb3e40, bb=0x7fff33c02d70, start=0x7ffff732a490 "H\213\025\311\331\022", initial_flags=0, 
    for_trace=false, unmangled_ilist=0x0) at /home/bruening/dr/git/src/core/arch/interp.c:5041
#19 0x00007ffff7deb2d6 in build_basic_block_fragment (dcontext=0x7fff33bb3e40, start=0x7ffff732a490 "H\213\025\311\331\022", initial_flags=0, link=true, 
    visible=true, for_trace=false, unmangled_ilist=0x0) at /home/bruening/dr/git/src/core/arch/interp.c:5116
#20 0x00007ffff7c447e7 in d_r_dispatch (dcontext=0x7fff33bb3e40) at /home/bruening/dr/git/src/core/dispatch.c:214

$ nm ext/lib64/debug/libdrsyms.so | grep ' U ' | grep alloc
                 U calloc@@GLIBC_2.2.5
                 U dr_global_alloc
                 U malloc@@GLIBC_2.2.5
                 U realloc@@GLIBC_2.2.5

One way to solve would be to replace malloc in our copy of libelftc and rebuild it, using ld -wrap or the pre-processor or maybe a patch. It will work w/ standalone, and the recent static lib change will re-route DR allocs back to malloc.

[derekbruening]

Well as part of rebuilding all the libraries I pulled in a newer version of the libelftc sources, and it is now asserting in our test:

194: Failed to unmangle xmlRelaxNGParseImportRefs
194: Failed to unmangle xmlRelaxNGParseImportRefs
194: libelftc_dem_gnu3.c:2138: cpp_demangle_read_sname: Assertion `ddata->output.size > 0' failed.

Plus, I move the redirect routines to be cross-platform, and now Windows crashes using them:

00 000000de`0294e190 00000000`151ab2d4 dynamorio!common_heap_free+0x1f7 [d:\derek\dr\git\src\core\heap.c @ 4488] 
01 000000de`0294e340 00000000`1518c2f7 dynamorio!common_global_heap_free+0x84 [d:\derek\dr\git\src\core\heap.c @ 3470] 
02 000000de`0294e380 00000000`1522d4a7 dynamorio!global_heap_free+0x37 [d:\derek\dr\git\src\core\heap.c @ 3511] 
03 000000de`0294e3c0 00000000`1524efd3 dynamorio!redirect_free+0x37 [d:\derek\dr\git\src\core\loader_shared.c @ 997] 
04 000000de`0294e3f0 00007ff6`1f22d54d dynamorio!__wrap_free+0x13 [d:\derek\dr\git\src\core\lib\instrument.c @ 3254] 
05 000000de`0294e420 00007ff6`1f227da3 drsyms!vector_str_dest+0x2d [d:\derek\dr\libelftc\libelftc\libelftc_vstr.c @ 76] 
06 000000de`0294e450 00007ff6`1f222c00 drsyms!cpp_demangle_gnu3+0x233 [d:\derek\dr\libelftc\libelftc\libelftc_dem_gnu3.c @ 235] 
07 000000de`0294e540 00007ff6`1f1d519b drsyms!elftc_demangle+0x70 [d:\derek\dr\libelftc\libelftc\elftc_demangle.c @ 93] 
08 000000de`0294e570 00007ff6`1f1d167b drsyms!drsym_unix_demangle_symbol+0x9b [d:\derek\dr\git\src\ext\drsyms\drsyms_unix_common.c @ 651] 
09 000000de`0294e7c0 00007ff6`1f173c1f drsyms!drsym_demangle_symbol+0x6b [d:\derek\dr\git\src\ext\drsyms\drsyms_windows.c @ 1529] 
0a 000000de`0294e800 00007ff6`1f171c35 client_drsyms_test_dll!test_demangle_symbols+0x6f [d:\derek\dr\git\src\suite\tests\client-interface\drsyms-test.dll.cpp @ 1151] 
0b 000000de`0294f050 00007ff6`1f171091 client_drsyms_test_dll!test_demangle+0x15 [d:\derek\dr\git\src\suite\tests\client-interface\drsyms-test.dll.cpp @ 1201] 
0c 000000de`0294f080 00000000`1524676a client_drsyms_test_dll!dr_init+0x91 [d:\derek\dr\git\src\suite\tests\client-interface\drsyms-test.dll.cpp @ 95] 
0d 000000de`0294f0d0 00000000`150109a0 dynamorio!instrument_init+0x26a [d:\derek\dr\git\src\core\lib\instrument.c @ 773] 
0e 000000de`0294f170 00000000`1533f31e dynamorio!dynamorio_app_init+0x6e0 [d:\derek\dr\git\src\core\dynamo.c @ 680] 

Grrr, this was supposed to be easy.

Is the assert a bug? Or are we using the library incorrectly somehow? Should I just revert to the prior version? And bail on updating Windows to have direct calls? Without them it would go through the Rtl heap and be redirected there, right? Why doesn't that hit a problem? Is there sthg wrong in our wrappers?

[derekbruening]

The Windows crash is from strdup not being intercepted. Once I rebuild with that redirected, Windows hits the same assert as Linux (though it surfaces as an assert during a callback raised during assert reporting).

The assert is from libelftc r3531. That change does fix the failure to mangle _ZN10linked_ptrIN12CrxInstaller14WhitelistEntryEE4copyIS1_EEvPKS_IT_E in drsyms-test, but then asserts on _ZZN7WebCore19SVGAnimatedProperty20LookupOrCreateHelperINS_32SVGAnimatedStaticPropertyTearOffIbEEbLb1EE21lookupOrCreateWrapperEPNS_10SVGElementEPKNS_15SVGPropertyInfoERbE19__PRETTY_FUNCTION__ which worked previously. It does not look like a simple bug to quickly work around or fix. I think we have to revert back to r3530, rebuild all the libraries with that, and file a bug vs libelftc. The old graceful failure is much better than an assert that will turn into memory corruption in release build.

[derekbruening]

I filed https://sourceforge.net/p/elftoolchain/tickets/581/