search

DynamoRIO / drmemory

Memory Debugger for Windows, Linux, Mac, and Android
Other
2.44k stars 262 forks source link

[DrM 1.11.0/win64] 64-bit application does not load in DRMemory 64 #1917

Open avaneev opened 8 years ago

avaneev commented 8 years ago

There is a certain application http://www.hermannseib.com/programs/dvsthostx64.zip which does not load when I start it using this command line: D:\TOOLS\DrMemory\bin\drmemory.exe -- "C:\Program Files\VSTHost64\vsthost.exe" (drmemory 1.11.0-2) It simply exits, no GUI is displayed.

derekbruening commented 8 years ago

Please try the steps under Narrowing Down the Source of the Problem at https://github.com/DynamoRIO/drmemory/wiki/Debugging as well as trying debug build for diagnostic messages.

avaneev commented 8 years ago

With -debug I get such error message: ASSERT FAILURE (thread 6464): D:\drmemory_package\drsyscall\drsyscall_windows.c:506: false (sysnum table does not match wrapper)

Running via drrun.exe crashes the application.

derekbruening commented 8 years ago

It sounds like an unsupported version of Windows. What version is it? Does drrun -debug complain about the version?

Which syscall doesn't match the wrapper? This should be at the end of the -debug global..log file.

I would expect an up-front failure like this to happen to every application, not just a particular one: but you've filed the issue on this one application, implying that other applications work? Does "hello, world" work when compiled as 64-bit? What about 32-bit?

avaneev commented 8 years ago

Windows 7 64 bit. drrun -debug does not complain about version. Simple "hello world" works fine in 64-bit.

Dr.memory 32 works with most applications.

WARNING: sysnum table 0xa != wrapper 0x1334ca for NtSetInformationThread ASSERT FAILURE (thread 6576): D:\drmemory_package\drsyscall\drsyscall_windows.c:506: false (sysnum table does not match wrapper)

derekbruening commented 8 years ago

0x1334ca?? If invasive software were hooking the wrappers it might fail to get a number, but to get a strange number like that is odd. There's something weird going on in that process. What is the disassembly of ntdll!NtSetInformationThread for that app?

avaneev commented 8 years ago

Well, I have no experience to debug this in any deeper way.