Crash running under drmemory

[derekbruening]

From pedro.la...@gmail.com on August 07, 2014 11:31:43

Running our program under drmemory with no arguments results in the "DynamoRIO Notice" in attachment dynamorio_notice_01.png.

The program does not crash when run normally.

We are using drmemory 1.7.0-5 on Windows Server 2008 R2 Standard SP1 64 bits. The programa is 32 bits.

With the -light argument the run results in the "DynamoRIO Notice" in attachment dynamorio_notice_02.png.

With the -leaks_only -no_count_leaks -no_track_allocs arguments the run completes normally with no "DynamoRIO Notice". Also, it finds no errors.

With the -debug -dr_debug -pause_at_assert arguments the run results in the "DynamoRIO Notice" in attachment dynamorio_notice_03.png.

Attachment: dynamorio_notice_01.png dynamorio_notice_02.png dynamorio_notice_03.png

Original issue: http://code.google.com/p/drmemory/issues/detail?id=1603

[derekbruening]

From zhao...@google.com on August 18, 2014 19:49:54

$ ./bin/symquery.exe -e ./bin/debug/drmemorylib.dll -f -a 1352e5 134d75 13454b 17e208 1a710b 1a3f14 19b8b4 1979ea 196b01 3238f 34bbf 19805 25330 iterator_lock+0x25 d:\drmemory_package\common\alloc_replace.c:560+0xc alloc_replace_overlaps_region+0x825 d:\drmemory_package\common\alloc_replace.c:2223+0xb alloc_replace_overlaps_delayed_free+0x1b d:\drmemory_package\common\alloc_replace.c:2259+0x18 overlaps_delayed_free+0x48 d:\drmemory_package\drmemory\alloc_drmem.c:1026+0x11 report_heap_info+0x234b d:\drmemory_package\drmemory\report.c:2308+0x1e print_error_to_buffer+0x8014 d:\drmemory_package\drmemory\report.c:2801+0x30 print_error_report+0x54 d:\drmemory_package\drmemory\report.c:2439+0x1b report_error+0xeda d:\drmemory_package\drmemory\report.c:2644+0x28 report_unaddressable_access+0xb31 d:\drmemory_package\drmemory\report.c:2870+0xf handle_mem_ref_internal+0x43ff d:\drmemory_package\drmemory\readwrite.c:4694+0x2b check_mem_opnd+0x129f d:\drmemory_package\drmemory\readwrite.c:4796+0x1ae slow_path_with_mc+0x46f5 d:\drmemory_package\drmemory\readwrite.c:3444+0x52 slow_path+0x50 d:\drmemory_package\drmemory\readwrite.c:3518+0x18

Seems a problem of heap work in replaced allocator.

Could you please provide a minimized program/code so that we can reproduce the problem?

[derekbruening]

From zhao...@google.com on August 18, 2014 19:58:35

$ ./bin/symquery.exe -e ./bin/release/drmemorylib.dll -f -a 34eab 3ca93 alloc_replace_overlaps_delayed_free+0x1b d:\drmemory_package\common\alloc_replace.c:2259+0x18 overlaps_delayed_free+0x43 d:\drmemory_package\drmemory\alloc_drmem.c:1027+0x0

Similar callstack from release build.

[derekbruening]

From pedro.la...@gmail.com on September 04, 2014 12:54:44

I have just seen zhao's comments; I thought email notifications were in place, but I was wrong. I am still working on analyzing that same program, but it is a large Windows Service, linking to ATL, openssl and even APR. I'll see what I can do.

[derekbruening]

From zhao...@google.com on September 04, 2014 13:10:25

Can you also try if -no_replace_malloc to see if it works?

[derekbruening]

From pedro.la...@gmail.com on September 09, 2014 11:01:01

I have reduced my case to a smaller program, unfortunately still dependent on a number of third party components -- log4cxx, apr, openssl etc.

With this program I have noticed that the program crashes while terminating.

Experimenting with a very large loop I realized that the program goes on normally until I decide to use drmemory.exe -nudge to get a partial result, when it then crashes.

Running [ drmemory.exe -no_replace_malloc ] with this smaller programa or the larger windows service does not lead to a crash, neither when the program terminates nor when I -nudge it.

It seems there are lots of UNADDRESSABLE ACCESSES or similar problems with the RAND mechanism of openssl, in particular the RAND_poll function. It also seems this is known by Valgrind users. Maybe this is relevant.