Fetching ntdll symbols for syscall auto-generation failing

DynamoRIO / drmemory

Memory Debugger for Windows, Linux, Mac, and Android

Other

2.43k stars 261 forks source link

Fetching ntdll symbols for syscall auto-generation failing #2232

Open Omegastick opened 4 years ago

Omegastick commented 4 years ago

Similar to #2126, I think, but for a different version (1903).

When running Dr Memory, I get the following output:

~~Dr.M~~ Dr. Memory version 2.2.0
~~Dr.M~~ Running "build\Debug\SingularityTrainer.exe"
~~Dr.M~~ Running on an unsupported operating system version: WinVer=105;Rel=1903;Build=18362;Edition=Core. Exiting to trigger auto-generation of system call information. Re-run with -ignore_kernel to attempt to continue instead.
WARNING: Running on an unsupported operating system version.  Attempting to auto-generate system call information...
drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable
Symbol cache directory is "D:\Programs\DrMemory\drmemory\logs\symcache"
Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #0
Ignoring local _NT_SYMBOL_PATH in next attempt.
Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #1
Ignoring local _NT_SYMBOL_PATH in next attempt.
Failed to fetch symbols for C:\WINDOWS\system32\ntdll.dll: error 1
~~Dr.M~~ WARNING: application exited with abnormal code 0xc000a004

derekbruening commented 4 years ago

1903 is supported. This is a failure to load symbols which is how DrMemory now supports the frequent Win10 major version updates.

Xref https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/drmemory-users/ywiBQ-LDYa0/fs8xW_nCCQAJ

Does your machine not have outside network access?

Please run with -vvv.

Omegastick commented 4 years ago

My internet is working fine on this machine, I'm not sure why it can't download the symbols.

Running with -vvv prints the following, then hangs (running without -vvv doesn't hang, it just prints the messages in my original issue within a few seconds).

INFO: targeting application: "C:\Users\Omega\prog\bots\build\Debug\SingularityTrainer.exe"
INFO: app cmdline: "build\Debug\SingularityTrainer.exe"
INFO: logdir is "D:\Programs\DrMemory\drmemory\logs"
INFO: symcache_dir is "D:\Programs\DrMemory\drmemory\logs\symcache"
drfront_set_client_symbol_search_path: setting D:\Programs\DrMemory\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\Programs\DrMemory\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\Programs\DrMemory\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
INFO: Setting TZ to Tokyo Standard Time for i#2164 workaround
INFO: DynamoRIO configuration directory is C:\Users\Omega/dynamorio
INFO: configuring SingularityTrainer.exe pid=35068 dr_ops="-disable_traces -bb_single_restore_prefix -max_bb_instrs 256 -vm_size 256M -no_enable_reset -no_vm_base_near_app -no_early_inject -logdir `D:\Programs\DrMemory\drmemory\logs\dynamorio` "
INFO: configuring client "D:\Programs\DrMemory\bin64\release\drmemorylib.dll" ops="-logdir `D:\Programs\DrMemory\drmemory\logs` -symcache_dir `D:\Programs\DrMemory\drmemory\logs\symcache` -lib_blacklist `C:\WINDOWS*.d??,C:\Program Files\Common Files\Microsoft Shared*.d??
,C:\Program Files (x86)\Common Files\Microsoft Shared*.d??` -resfile 35068 "
INFO: waiting for app to exit...
~~Dr.M~~ Dr. Memory version 2.2.0
~~Dr.M~~ Running "build\Debug\SingularityTrainer.exe"
~~Dr.M~~ Running on an unsupported operating system version: WinVer=105;Rel=1903;Build=18362;Edition=Core. Exiting to trigger auto-generation of system call information. Re-run with -ignore_kernel to attempt to continue instead.
WARNING: Running on an unsupported operating system version.  Attempting to auto-generate system call information...
drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable
Symbol cache directory is "D:\Programs\DrMemory\drmemory\logs\symcache"
drfront_set_client_symbol_search_path: setting D:\Programs\DrMemory\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\Programs\DrMemory\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\Programs\DrMemory\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #0

I left it for a few minutes, but maybe it's just a big download. I'll try again and leave it overnight (my internet is pretty fast here though, so it shouldn't be a download speed problem).

I'll look into getting the symbols manually.

lamonosa commented 4 years ago

Hi, is there any progress about the issue? I'm facing the same problem, for version 2301

derekbruening commented 4 years ago

I can't reproduce this and it's not clear what is causing the problem. If you could walk through in the debugger as discussed below:

The symbols are downloaded from the Microsoft symbol server with this code:

https://github.com/DynamoRIO/drmemory/blob/master/drsyscall/pdb2sysfile.cpp#L482

https://github.com/DynamoRIO/dynamorio/blob/master/libutil/dr_frontend_win.c#L505

The error without -vvv:

Failed to fetch symbols for C:\WINDOWS\system32\ntdll.dll: error 1

Error code 1 is unfortunately the vague DRFRONT_ERROR. Looking at the code, it is probably a failure in sym_load_module_func which should point at dbghelp.dll!SymLoadModuleExW, whose docs are here: https://docs.microsoft.com/en-us/windows/win32/api/dbghelp/nf-dbghelp-symloadmoduleex

I would suggest walking through that code in a debugger to see where it fails and to look at the Windows GetLastError.

frontend.c for -vvv calls drfront_set_verbose(3) so the NOTIFY would show up if reached: so it seems to be hanging in the dbghelp.dll!SymLoadModuleExW call which seems odd when the same call, which is no different without -vvv, returned failure before??

gamestarjoren commented 4 years ago

Hey, dude, I'm facing the same problem too. Here's the log:

Dr. Memory version 2.3.0 build 1 built on Feb  6 2020 06:07:09
Windows version: WinVer=105;Rel=1909;Build=18363;Edition=Professional
Dr. Memory results for pid 2196: "ConsoleApplication1.exe"
Application cmdline: "C:\Users\11471\source\repos\ConsoleApplication1\x64\Debug\ConsoleApplication1.exe"
Recorded 118 suppression(s) from default C:\Program Files (x86)\Dr. Memory\bin64\suppress-default.txt
System call information is missing for this operating system: WinVer=105;Rel=1909;Build=18363;Edition=Professional. Restarting to trigger auto-generation of system call information. Re-run with -ignore_kernel to attempt to continue instead.
Syscall mismatch detected.  Running on unknown kernel version!

with -vvv,:

Symbol cache directory is "C:\Users\11471\AppData\Roaming\Dr. Memory\symcache"
drfront_set_client_symbol_search_path: setting C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #0
drfront_fetch_module_symbols: SymLoadModuleEx C:\WINDOWS\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\WINDOWS\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #1
drfront_fetch_module_symbols: SymLoadModuleEx C:\WINDOWS\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\WINDOWS\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols
Failed to fetch symbols for C:\WINDOWS\system32\ntdll.dll: error 1
INFO: fetching symbols
~~Dr.M~~ WARNING: application exited with abnormal code 0xc000a004

derekbruening commented 4 years ago

That's a different error from what the poster of the issue is hitting:

  failed to fetch pdb for C:\WINDOWS\system32\ntdll.dll, exports only

Is there no network access on that machine?

Does symbol loading work with the separate MS tools? E.g., something like this, depending on where you have symchk.exe:

c:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86\symchk.exe /v /od /os /ot /ov /ob c:\WINDOWS\system32\ntdll.dll /s 'srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols'

lefty2 commented 3 years ago

I think there are two problems here, first dr. memory needs to be put on the firewall whitelist, so it can download the symbols. The second problem is the crash. There seems to be no solution to that

derekbruening commented 3 years ago

The second problem is the crash.

The exit code 0xc000a004 is STATUS_INVALID_KERNEL_INFO_VERSION and is deliberately returned here by Dr. Memory, so it is part of the same issue with the missing symbols.

lefty2 commented 3 years ago

If you allow it to download the symbols, then that error goes away, but you then have a different error not related to missing symbols:

Dr. Memory version 2.3.0 Running "c:\backup\warlord\Warlord\debug\MercTactics.exe" Using system call file C:\Users\Lefty\AppData\Roaming\Dr. Memory\symcache\syscalls_wow64.txt <Application c:\backup\warlord\Warlord\debug\MercTactics.exe (11796). Dr. Memory internal crash at PC 0x6f86d99b. Please report this at http://drmemory.org/issues along with the results of running '-debug -dr_debug'. Program aborted. 0xc0000005 0x00000000 0x6f86d99b 0x6f86d99b 0x00000003 0x00000000 Base: 0x6f7c0000 Registers: eax=0x00000000 ebx=0x0019f688 ecx=0x26c1de54 edx=0x00000000 esi=0x1a116b90 edi=0x1a116b88 esp=0x0019f6a8 ebp=0x0019f75c eflags=0x000 2.3.0-1-(Feb 6 2020 06:09:03) WinVer=105;Rel=1903;Build=18362;Edition=Enterprise -no_dynamic_options -disasm_mask 8 -logdir 'C:\Users\Lefty\AppData\Roaming\Dr. Memory\dynamorio' -client_lib 'C:\Program Files (x86)\Dr. Memory\bin\release\drmemorylib.dll;0;-visual_studio -logdir C:\Users\Lefty\AppData\Roaming\Dr. Memory -symcache_dir C:\Users\Lefty\AppData\Roaming\Dr. Memory\symcache -lib_blacklist 0x0019f75c 0x6f846460> WARNING: application exited with abnormal code 0xffffffff

Application c:\backup\warlord\Warlord\debug\MercTactics.exe (11796). Dr. Memory internal crash at PC 0x6f86d99b. Please report this at http://drmemory.org/issues along with the results of running '-debug -dr_debug'. Program aborted. 0xc0000005 0x00000000 0x6f86d99b 0x6f86d99b 0x00000003 0x00000000 Base: 0x6f7c0000 Registers: eax=0x00000000 ebx=0x0019f688 ecx=0x26c1de54 edx=0x00000000 esi=0x1a116b90 edi=0x1a116b88 esp=0x0019f6a8 ebp=0x0019f75c eflags=0x000 2.3.0-1-(Feb 6 2020 06:09:03) WinVer=105;Rel=1903;Build=18362;Edition=Enterprise -no_dynamic_options -disasm_mask 8 -logdir 'C:\Users\Lefty\AppData\Roaming\Dr. Memory\dynamorio' -client_lib 'C:\Program Files (x86)\Dr. Memory\bin\release\drmemorylib.dll;0;-visual_studio -logdir C:\Users\Lefty\AppData\Roaming\Dr. Memory -symcache_dir C:\Users\Lefty\AppData\Roaming\Dr. Memory\symcache -lib_blacklist 0x0019f75c 0x6f846460

derekbruening commented 3 years ago

you then have a different error not related to missing symbols:

Please file a separate issue rather than lumping into this one which is about symbols. Please fill out the bug template for that new issue (supposed to show up on any new bug but a Github bug prevents it if you're not logged in when you click the button) by using this link: https://github.com/DynamoRIO/drmemory/issues/new?assignees=&labels=&template=bug_report.md&title=

mrozo94 commented 3 years ago

I can confirm that issue is present on the newest DrMemory & Win10. Luckly you can ignore it with "ignore_kernel"

DrMemory - 2.3.18696 Win10 - 10.0.19042

INFO: targeting application: "D:\cygwin64\home\Mrozo_5GPTN\5GPTN-MRO\5GPTN-Mro.exe"
INFO: app cmdline: "./5GPTN-Mro.exe" "0" "../5GPTN-TestFiles/topologiesMgr/ring_RND_3r3"
INFO: logdir is "D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs"
INFO: symcache_dir is "D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache"
drfront_set_client_symbol_search_path: setting D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
INFO: DynamoRIO configuration directory is C:\Users\Mrozo_5GPTN/dynamorio
INFO: configuring 5GPTN-Mro.exe pid=2376 dr_ops="-disable_traces -bb_single_restore_prefix -max_bb_instrs 256 -vm_size 256M -no_enable_reset -no_early_inject -logdir `D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\dynamorio` "
INFO: configuring client "D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\bin64\release\drmemorylib.dll" ops="-logdir `D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs` -symcache_dir `D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\dr
memory\logs\symcache` -lib_blocklist_default `C:\Windows*.d??,C:\Program Files\Common Files\Microsoft Shared*.d??,C:\Program Files (x86)\Common Files\Microsoft Shared*.d??` -resfile 2376 "
INFO: waiting for app to exit...
~~Dr.M~~ Dr. Memory version 2.3.18696
~~Dr.M~~ Running "./5GPTN-Mro.exe 0 ../5GPTN-TestFiles/topologiesMgr/ring_RND_3r3"
~~Dr.M~~ System call information is missing for this operating system: WinVer=105;Rel=2009;Build=19042;Edition=Professional. Restarting to trigger auto-generation of system call information. Re-run with -ignore_kernel to attempt to continue instead.
WARNING: System call information is missing for this operating system version. Attempting to auto-generate system call information...
drsys_find_sysnum_libs: C:\Windows\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\win32u.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\ntdll.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\kernelbase.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\kernel32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\gdi32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\imm32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\user32.dll is readable
drsys_find_sysnum_libs: C:\Windows\system32\win32u.dll is readable
Symbol cache directory is "D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache"
drfront_set_client_symbol_search_path: setting D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\Windows\system32\ntdll.dll", attempt #0
drfront_fetch_module_symbols: SymLoadModuleEx C:\Windows\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\Windows\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\Windows\system32\ntdll.dll", attempt #1
drfront_fetch_module_symbols: SymLoadModuleEx C:\Windows\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\Windows\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols as the local store
drfront_set_client_symbol_search_path: returning srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*D:\cygwin64\home\Mrozo_5GPTN\DrMemory-Windows-2.3.18696\drmemory\logs\symcache\symbols*http://msdl.microsoft.com/download/symbols
Failed to fetch symbols for C:\Windows\system32\ntdll.dll: error 1
INFO: fetching symbols
~~Dr.M~~ WARNING: application exited with abnormal code 0xc000a004

This issue has tag "Status-NeedInfo" - let me know how could I provide additional info that can be usefull?

derekbruening commented 3 years ago

This issue has tag "Status-NeedInfo" - let me know how could I provide additional info that can be usefull?

We have not been able to reproduce any of these symbol loading failures. As you can see above (https://github.com/DynamoRIO/drmemory/issues/2232#issuecomment-642143003, ,https://github.com/DynamoRIO/drmemory/issues/2232#issuecomment-666340600) I suggested several routes to help figure out what is going on: walking through in the debugger; seeing whether symchk by itself works. Is it a firewall or other local network issue specific to your setup that is stopping symbol downloading?

mrozo94 commented 3 years ago

Is it a firewall or other local network issue specific to your setup that is stopping symbol downloading?

Even with the firewall disabled, I can't download symbols. I'll try to check other things you mentioned in some free time.

brandonros commented 2 years ago

drfront_set_client_symbol_search_path: returning srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*http://msd
l.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*h
ttp://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\Windows\system32\ntdll.dll", attempt #0
drfront_fetch_module_symbols: SymLoadModuleEx C:\Windows\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\Windows\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols as the local sto
re
drfront_set_client_symbol_search_path: returning srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*http://msd
l.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*h
ttp://msdl.microsoft.com/download/symbols
Fetching symbols for "C:\Windows\system32\ntdll.dll", attempt #1
drfront_fetch_module_symbols: SymLoadModuleEx C:\Windows\system32\ntdll.dll => 0x180000000
  failed to fetch pdb for C:\Windows\system32\ntdll.dll, exports only
Ignoring local _NT_SYMBOL_PATH in next attempt.
drfront_set_client_symbol_search_path: setting C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols as the local sto
re
drfront_set_client_symbol_search_path: returning srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*http://msd
l.microsoft.com/download/symbols
drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\Brandon\Downloads\DrMemory-Windows-2.4.18900\DrMemory-Windows-2.4.18900\drmemory\logs\symcache\symbols*h
ttp://msdl.microsoft.com/download/symbols
Failed to fetch symbols for C:\Windows\system32\ntdll.dll: error 1
INFO: fetching symbols
~~Dr.M~~ WARNING: application exited with abnormal code 0xc000a004

DanielBernhard commented 10 months ago

I have some more information about this issue: We can reproduce this issue as well when running with jenkins. However doing a manual run works fine. As I use the same user in both cases IMO it must be a difference between starting the run from a service or from console.

System: Windows 10 (WinVer=105;Rel=2009;Build=19045;Edition=Enterprise) DrMemory 2.5.0 (Note: 2.6.0 crashes on my machine) Working with 32-bit code

Console with -vvv (when starting as service): drmemory.exe -quiet -batch -vvv -logdir DrMemory -suppress 'C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\Build\DrMemorySuppression.txt' -- 'Win32\TestIoHandling.exe' --gtest_output=xml:Win32/test_detail.xml INFO: targeting application: "C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\Win32\TestIoHandling.exe" INFO: app cmdline: "Win32\TestIoHandling.exe" "--gtest_output=xml:Win32/test_detail.xml" INFO: logdir is "C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory" INFO: symcache_dir is "C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache" drfront_set_client_symbol_search_path: setting C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols as the local store drfront_set_client_symbol_search_path: returning srv*C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols*http://msdl.microsoft.com/download/symbols drfront_set_symbol_search_path: set symbol search path to srv*C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols*http://msdl.microsoft.com/download/symbols INFO: DynamoRIO configuration directory is C:\Users\intcrddredsrv/dynamorio INFO: configuring TestIoHandling.exe pid=9248 dr_ops="-disable_traces -bb_single_restore_prefix -max_bb_instrs 256 -vm_size 256M -no_enable_reset -no_early_inject -msgbox_mask 0 -stderr_mask 0 -msgbox_mask 0 -logdirC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\dynamorio" INFO: configuring client "C:\Jenkins\workspace\DSW_NB_Trunk\Common\Environment\Tools\DrMemory\bin\release\drmemorylib.dll" ops="-quiet -suppressC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\Build\DrMemorySuppression.txt-logdirC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory-symcache_dirC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache-lib_blocklist_defaultC:\WINDOWS.d??,C:\Program Files (x86)\Common Files\Microsoft Shared.d??,C:\Program Files (x86)\Common Files\Microsoft Shared.d??` -resfile 9248 " INFO: waiting for app to exit... drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\ntdll.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\kernelbase.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\kernel32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\gdi32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\imm32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\user32.dll is readable drsys_find_sysnum_libs: C:\WINDOWS\system32\win32u.dll is readable Symbol cache directory is "C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache" drfront_set_client_symbol_search_path: setting C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols as the local store drfront_set_client_symbol_search_path: returning srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbolshttp://msdl.microsoft.com/download/symbols drfront_set_symbol_search_path: set symbol search path to srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbolshttp://msdl.microsoft.com/download/symbols Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #0 drfront_fetch_module_symbols: SymLoadModuleEx C:\WINDOWS\system32\ntdll.dll => 0x4b280000 failed to fetch pdb for C:\WINDOWS\system32\ntdll.dll, exports only Ignoring local _NT_SYMBOL_PATH in next attempt. drfront_set_client_symbol_search_path: setting C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols as the local store drfront_set_client_symbol_search_path: returning srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbolshttp://msdl.microsoft.com/download/symbols drfront_set_symbol_search_path: set symbol search path to srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbolshttp://msdl.microsoft.com/download/symbols Fetching symbols for "C:\WINDOWS\system32\ntdll.dll", attempt #1 drfront_fetch_module_symbols: SymLoadModuleEx C:\WINDOWS\system32\ntdll.dll => 0x4b280000 failed to fetch pdb for C:\WINDOWS\system32\ntdll.dll, exports only Ignoring local _NT_SYMBOL_PATH in next attempt. drfront_set_client_symbol_search_path: setting C:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols as the local store drfront_set_client_symbol_search_path: returning srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbolshttp://msdl.microsoft.com/download/symbols drfront_set_symbol_search_path: set symbol search path to srvC:\Jenkins\workspace\DSW_NB_Trunk\Common\Implementation\IoHandling\Build\DrMemory\symcache\symbols*http://msdl.microsoft.com/download/symbols Failed to fetch symbols for C:\WINDOWS\system32\ntdll.dll: error 1 `

strawgate commented 10 months ago

Im hitting this issue.

I tried to run: c:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86\symchk.exe /v /od /os /ot /ov /ob c:\WINDOWS\system32\ntdll.dll /s 'srv*C:\Users\11471\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols' but had to switch the single quotes on the /s parameter to double quotes.

I then get an HTTP to HTTPs redirect error:

SYMSRV:  HTTPGET: /download/symbols/index2.txt
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  HTTPGET: /download/symbols/ntdll.pdb/3359C2C66D31CC534A1D2A8C5DCCC4971/ntdll.pdb
SYMSRV:  HttpSendRequest: 800C2F07 - ERROR_INTERNET_HTTP_TO_HTTPS_ON_REDIR
SYMSRV:  RESULT: 0x800C2F07
DBGHELP: ntdll - no symbols loaded

Changing it to https makes it work just fine

The Drmemory log seems to indicate that it's pointed at the http url: drfront_set_symbol_search_path: set symbol search path to srv*C:\Users\bill_easton\AppData\Roaming\Dr. Memory\symcache\symbols*http://msdl.microsoft.com/download/symbols