Open postiffm opened 8 years ago
postiffm
commented
8 years ago I should also say that I am running on 64-bit windows, and the app was compiled with mingw64/bin/g++ in the msys2 environment. I have not tried the 32-bit version of my code.
derekbruening
commented
8 years ago First, please confirm that these are msys utilities and not cygwin utilties.
Next, please try the steps listed at https://github.com/DynamoRIO/drmemory/wiki/Debugging under "Narrowing Down the Source of the Problem" to help identify where the problem is.
postiffm
commented
8 years ago Step one in narowing down the source of the problem. drmemory -dr_debug .\cp.exe --version
---------------------------
Dr. Memory Notice: C:\Program Files\Bibledit-4.9.3\editor\bin\cp.exe(10168)
---------------------------
Application C:\Program Files\Bibledit-4.9.3\editor\bin\cp.exe (10168). Dr. Memory internal crash at PC 0x000000007725cf27. Please report this at http://drmemory.org/issues. Program aborted.
0x80000001 0x00000000 0x000000007725cf27 0x000000007725cf27 0x0000000000000001 0x00000000ffff8000
Base: 0x0000000015000000
Registers: eax=0x00000000000014b8 ebx=0x0000000011dc0000 ecx=0x000000008102ad90 edx=0x00000000809d1df4
esi=0x000000000001f000 edi=0x000000008072d2ca esp=0x000000008072c9a0 ebp=0x000000008072e120
r8 =0x000000008102ad90 r9 =0x00000000809d108a r10=0x000000008072b000 r11=0x00000000ffff8000
r12=0x0000000081069d30 r13=0x0000000000000000 r14=0x0000000081069d78 r15=0x0000000000030000
eflags=0xabababab00010286
1.10.0-2-(Mar 7 2016 23:05:08) win61
-no_dynamic_options -disasm_mask 8 -logdir 'C:\Users\postiffm\AppData\Roaming\Dr. Memory\dynamorio' -client_lib 'C:\Program Files (x86)\Dr. Memory\bin64\release\drmemorylib.dll;0;-logdir `C:\Users\postiffm\AppData\Roaming\Dr. Memory` -symcache_dir `C:\Users\postiffm\AppData\Roaming\Dr. Memory\symcache` -lib_blacklist `C:
0x000000008072e120 0x0000000000000000
---------------------------
OK
---------------------------From logfile:
Dr. Memory version 1.10.0 build 2 built on Mar 7 2016 23:05:08 Dr. Memory results for pid 10168: "cp.exe" Application cmdline: ".\cp.exe --version" Recorded 117 suppression(s) from default C:\Program Files (x86)\Dr. Memory\bin64\suppress-default.txt
.\cp.exe --version
cp (GNU coreutils) 8.24 Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Torbj"orn Granlund, David MacKenzie, and Jim Meyering.
I installed this in from a msys2 shell from the coreutils package:
$ pacman -Ss coreutils msys/coreutils 8.24-1 (base) [installed] The basic file, shell and text manipulation utilities of the GNU operating system
derekbruening
commented
8 years ago Please put triple backtics around the output as I've edited some of the above to do (the github wiki is rather ridiculous and almost any non-alphanum char turns into crazy formatting)
postiffm
commented
8 years ago FAILS: >drmemory -dr_debug -no_count_leaks .\cp.exe --version FAILS: >drmemory -dr_debug -light .\cp.exe --version FAILS: >drmemory -dr_debug -leaks_only .\cp.exe --version FAILS: >drmemory -dr_debug -leaks_only -no_count_leaks .\cp.exe --version FAILS: >drmemory -dr_debug -leaks_only -no_count_leaks -no_track_allocs .\cp.exe --version
Each of the above also fails if I remove the -dr_debug flag.
WORKS: >"C:\Program Files (x86)\Dr. Memory\dynamorio\bin64\drrun" -- .\cp.exe --version cp (GNU coreutils) 8.24
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Torbj"orn Granlund, David MacKenzie, and Jim Meyering.Does Dr. Memory give any debug info if you enable its debug version (you were only enabling DynamoRIO debug), by running with -debug?
postiffm
commented
8 years ago Another clue. When I drag-n-drop ont to the Dr. Memory icon, I believe the starting or working directory is different than if I run on the command line directly from C:\Program Files...\bibledit-gtk.exe. I think this makes a difference because my program dynamically finds the file utilities that are available on the system. When running from the command line, it finds cp.exe in the current dir. This fails. When running in drag-n-drop mode, it finds the copy shell command and uses that.
This explains why drag-n-drop works better. It doesn't explain why drag-n-drop produces no log files in C:\Users\my-username\AppData\Roaming\Dr. Memory.
postiffm
commented
8 years ago FAILS: >drmemory -debug -no_count_leaks .\cp.exe --version
~~Dr.M~~ Dr. Memory version 1.10.0
~~Dr.M~~ (Uninitialized read checking is not yet supported for 64-bit)
~~Dr.M~~ Running ".\cp.exe --version"
~~Dr.M~~ ASSERT FAILURE (thread 12320): d:\drmemory_package\common\alloc.c:4781:
pt->valloc_type == MEM_DECOMMIT || !is_in_heap_region(base) (heap region tracki
ng bug)~~Dr.M~~ WARNING: application exited with abnormal code 0xffffffffFAILS SAME WAY: >drmemory -debug -light .\cp.exe --version FAILS SAME WAY: >drmemory -debug -leaks_only .\cp.exe --version FAILS SAME WAY: >drmemory -debug -leaks_only -no_count_leaks .\cp.exe --version
FAILS DIFFERENTLY: >drmemory -debug -leaks_only -no_count_leaks -no_track_allocs .\cp.exe --version
It does not throw an assertion violation. Instead, it shows the following in a pop-up window:
---------------------------
Dr. Memory Notice: C:\Program Files\Bibledit-4.9.3\editor\bin\cp.exe(2724)
---------------------------
Application C:\Program Files\Bibledit-4.9.3\editor\bin\cp.exe (2724). Dr. Memory internal crash at PC 0x000000007725cf27. Please report this at http://drmemory.org/issues. Program aborted.
0x80000001 0x00000000 0x000000007725cf27 0x000000007725cf27 0x0000000000000001 0x00000000ffff8000
Base: 0x0000000071000000
Registers: eax=0x00000000000014b8 ebx=0x0000000000000001 ecx=0x00000000810611b8 edx=0x0000000080e4a57c
esi=0x0000000080c7b140 edi=0x0000000080ccd61a esp=0x0000000080ccccf0 ebp=0x0000000080c33238
r8 =0x00000000810611b8 r9 =0x0000000080e49812 r10=0x0000000080ccb000 r11=0x00000000ffff8000
r12=0x0000000000000001 r13=0x0000000000000000 r14=0x0000000080ccee90 r15=0x000007fefd3560e8
eflags=0x0000000000010286
1.10.0-2-(Mar 7 2016 23:05:31) win61
-no_dynamic_options -logdir 'C:\Users\postiffm\AppData\Roaming\Dr. Memory\dynamorio' -client_lib 'C:\Program Files (x86)\Dr. Memory\bin64\debug\drmemorylib.dll;0;`-leaks_only` `-no_count_leaks` `-no_track_allocs` -logdir `C:\Users\postiffm\AppData\Roaming\Dr. Memory` -symcache_dir `C:\Users\postiffm\AppData\Roaming\Dr. M
0x0000000080c33238 0x000007fefd36f000
0x000007fefd350000 0x0000ffff00000004
---------------------------
OK
---------------------------If you run with -no_follow_children (leaving the child utility processes native) does the parent then work properly?
postiffm
commented
8 years ago drmemory -debug -no_follow_children bibledit-gtk.exe
For the 64-bit compiled version of the program, it starts and functions. No results.txt file is found. Error message is:
~~Dr.M~~ WARNING: unable to locate results file: can't open C:\Users\postiffm\Ap
pData\Roaming\Dr. Memory/resfile.11960 (code=2).
Dr. Memory failed to start the target application, perhaps due to
interference from invasive security software.
Try disabling other software or running in a virtual machine.For the 32-bit compiled version of the program (same source code), it starts and a results.txt is generated from Dr. Memory. The data contained therein makes sense.
[Edited from yesterday; I was running from the wrong path and so the program only partially functioned. All other results, including Dr. Memory problem, remains as it was.]
derekbruening
commented
8 years ago Looking at the description of msys2, if it does indeed contain cygwin code, that may provide some explanation of some of the problems: we do not currently support cygwin (due to too much weird behavior in cygwin1.dll). We have a check for "cygwin1.dll" today, but it looks like they build their own with a different name: msys-2.0.dll with the string description "Cygwin POSIX Emulation DLL".
derekbruening
commented
8 years ago 32-bit msys2 seems to work under Dr. Memory, but 64-bit has a number of problems seemingly related to strange stack behavior:
% ~/dr/releases/DynamoRIO-Windows-6.1.0-2/bin64/drrun -debug -- e:/msys64/usr/bin/ls.exe
<Starting application e:\msys64\usr\bin\ls.exe (5040)>
<Early threads found>
<Initial options = -no_dynamic_options -code_api -probe_api -stack_size 56K -max_elide_jmp 0 -max_elide_call 0 -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -pad_jmps_mark_no_trace >
<CURIOSITY : get_os_version() <= 50 in file d:\dynamorio_package\core\win32\syscall.c line 1923
version 6.1.0, build 2
-no_dynamic_options -code_api -probe_api -stack_size 56K -max_elide_jmp 0 -max_elide_call 0 -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -pad_jmps_mark_no_trace
0x00000000fffffff0 0x0000000000000000>
<CURIOSITY : (byte *)get_mcontext(dcontext)->xsp >= (byte *)((TEB *)((void *) __readgsqword(SELF_TIB_OFFSET))) && (byte *)get_mcontext(dcontext)->xsp < ((byte *)((TEB *)((void *) __readgsqword(SELF_TIB_OFFSET)))) + (4*1024) in file d:\dynamorio_package\core\win32\syscall.c line 1930
version 6.1.0, build 2
-no_dynamic_options -code_api -probe_api -stack_size 56K -max_elide_jmp 0 -max_elide_call 0 -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -pad_jmps_mark_no_trace
0x00000000fffffff0 0x0000000000000000>
<writing to executable region.>Both curiosities are in check_for_stack_free().
DR output to the console stops, presumably due to the app playing games with the console.
64-bit DrM w/ debug DR:
<CURIOSITY : (((0)) && get_mcontext(dcontext)->xsp >= (reg_t)dcontext->app_stack_limit) || (!((0)) && get_mcontext(dcontext)->xsp >= (reg_t)((void *) __readgsqword(BASE_STACK_TIB_OFFSET))) in file D:\derek\drmemory\git\src\dynamorio\core\dispatch.c line 937I can repro the assert:
% ~/drmemory/releases/DrMemory-Windows-1.10.0-2/bin64/drmemory -debug -- e:/msys64/usr/bin/ls.exe
~~Dr.M~~ Dr. Memory version 1.10.0
~~Dr.M~~ (Uninitialized read checking is not yet supported for 64-bit)
~~Dr.M~~ Running "e:/msys64/usr/bin/ls.exe"
~~Dr.M~~ ASSERT FAILURE (thread 1932): d:\drmemory_package\common\alloc.c:4781: pt->valloc_type == MEM_DECOMMIT || !is_in_heap_region(base) (heap region tracking bug)~~Dr.M~~ WARNING: application exited with abnormal code 0xffffffffAnd trying to debug that via -verbose 2 we have a crash:
Xref different DR curiosities above about stacks being freed strangely and about
the TEB stack fields.
0:000> .cxr @@(cxt)
rax=00000000000010d0 rbx=0000000000000286 rcx=0000000080e48440
rdx=0000000080e448f8 rsi=0000000000000000 rdi=0000000080d1dcd0
rip=0000000076f8d707 rsp=0000000080d1d5e8 rbp=00000000fffffff0
r8=000000008110acb8 r9=0000000000000000 r10=0000000080d1c000
r11=00000000ffffe000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010286
ntdll!_chkstk+0x37:
00000000`76f8d707 41c60300 mov byte ptr [r11],0 ds:00000000`ffffe000=00
0:000> kn
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 00000000`80d1d5e8 00000000`90c75f8f ntdll!_chkstk+0x37
01 00000000`80d1d600 00000000`90c7ac80 dbghelp+0x25f8f
02 00000000`80d1d610 00000000`90ca8fe3 dbghelp+0x2ac80
03 00000000`80d1d6e0 00000000`90c95199 dbghelp!ImagehlpApiVersionEx+0x9023
04 00000000`80d1d740 00000000`90c9985d dbghelp!SymEnumerateSymbolsW64+0x129
05 00000000`80d1d790 00000000`807acafc dbghelp!SymFromAddr+0x4d
06 00000000`80d1d7d0 00000000`807ac994 drmemorylib!drsym_lookup_address_local+0x15c [d:\derek\drmemory\git\src\dynamorio\ext\drsyms\drsyms_windows.c @ 680]
07 00000000`80d1d8b0 00000000`8060c3b6 drmemorylib!drsym_lookup_address+0x44 [d:\derek\drmemory\git\src\dynamorio\ext\drsyms\drsyms_windows.c @ 1495]
08 00000000`80d1d8e0 00000000`8060b768 drmemorylib!lookup_func_and_line+0x566 [d:\derek\drmemory\git\src\common\callstack.c @ 544]
09 00000000`80d1dce0 00000000`805fee3f drmemorylib!address_to_frame+0x2078 [d:\derek\drmemory\git\src\common\callstack.c @ 936]
0a 00000000`80d1e000 00000000`805fedba drmemorylib!print_address_common+0x6f [d:\derek\drmemory\git\src\common\callstack.c @ 966]
0b 00000000`80d1e300 00000000`80619601 drmemorylib!print_address+0x7a [d:\derek\drmemory\git\src\common\callstack.c @ 985]
0c 00000000`80d1e360 00000000`8068455f drmemorylib!print_callstack_to_file+0xc91 [d:\derek\drmemory\git\src\common\callstack.c @ 1972]
0d 00000000`80d1e510 00000000`806453a0 drmemorylib!report_callstack+0x4f [d:\derek\drmemory\git\src\drmemory\report.c @ 3630]
0e 00000000`80d1e550 00000000`8079a54b drmemorylib!event_pre_syscall+0x1570 [d:\derek\drmemory\git\src\drmemory\syscall.c @ 485]
0f 00000000`80d1e840 00000000`152a7379 drmemorylib!drmgr_presyscall_event+0xab [d:\derek\drmemory\git\src\dynamorio\ext\drmgr\drmgr.c @ 1300]
0:000> !teb
TEB at 000007fffffdc000
ExceptionList: 0000000000000000
StackBase: 0000000080d1f000
StackLimit: 00000000fffff000
BaseAddress: 0000000080d02000
RegionSize: 000000000000f000
State: 00002000 MEM_RESERVE
Type: 00020000 MEM_PRIVATE
BaseAddress: 0000000080d11000
RegionSize: 0000000000001000
State: 00001000 MEM_COMMIT
Protect: 00000104 PAGE_READWRITE + PAGE_GUARD
Type: 00020000 MEM_PRIVATE
BaseAddress: 0000000080d12000
RegionSize: 000000000000d000
State: 00001000 MEM_COMMIT
Protect: 00000004 PAGE_READWRITE
Type: 00020000 MEM_PRIVATE
BaseAddress: 0000000080d1f000
RegionSize: 0000000000002000
State: 00002000 MEM_RESERVE
Type: 00020000 MEM_PRIVATE
BaseAddress: 0000000080d21000
RegionSize: 0000000000006000
State: 00001000 MEM_COMMIT
Protect: 00000004 PAGE_READWRITE
Type: 00020000 MEM_PRIVATE
BaseAddress: 0000000080d27000
RegionSize: 000000000000a000
State: 00002000 MEM_RESERVE
Type: 00020000 MEM_PRIVATEIt does not look simple so for now we may want to put a check and print a message about not supporting it yet until someone can sit down and figure out what is going on.
derekbruening
commented
8 years ago 0d0b0bc6edab2c007390cc55a610098989b455a8 added a check for msys-2.0.dll and issues an unsupported message
I have a GTK program that I'm trying to run under Dr. Memory called bibledit-gtk. It uses a number of file utilities like cp, mv, rm, gzip, etc.
When I run drmemory -dr_debug bibledit-gtk.exe from a cmd shell, the program starts but does not run it correctly because, I believe, it crashes and does not permit the file utilities to do their job. It gives me the fatal errors I have pasted below.
Strangely, when I drag the executable onto the desktop Dr. Memory icon, the program seems to run and function properly.
In neither case is there a log file afterward for me to examine.
Thanks for any help you can provide.