LOST_CONTROL (security-win32.outputdebugstring.exe) not coming back from native mode with OutputDebugString

[Simorfo]

Is is not a CRASH or a ASSERT, at some point(with a syscall), dynamorio loses control so I chose LOST_CONTROL

With version 6.2.0-2 of DynamoRio The latest build does not solve the problem

On Windows 7, with a 32 bit application, security-win32.outputdebugstring.exe, a simple application using OutputDebugString function

I run it with (no client) C:\rio\bin32\drrun.exe -debug -loglevel 4 -- security-win32.outputdebugstring.exe

The expected output are some lines ending with "end of debug" This is almost ok. But the logs show that dynamorio does not come back from native after calling OutputDebugString function.

Here is the code for security-win32.outputdebugstring.exe

static void
debuggee(void)
{
    OutputDebugString("debug string test");
    print("after OutputDebugString\n");
}

static void
debugger(void)
{
    DEBUG_EVENT DebugEvent;
    STARTUPINFO si;
    PROCESS_INFORMATION processInfo;
    LPTSTR cmdline = GetCommandLine();

    ZeroMemory( &si, sizeof(si) );
    si.cb = sizeof(si);

    if (CreateProcess(cmdline, NULL, 0, 0, TRUE, DEBUG_ONLY_THIS_PROCESS, NULL, NULL, &si, &processInfo) == NULL) {
        print("fail CreateProcess\n");
        return;
    }
    print("after CreateProcess\n");

    while (WaitForDebugEvent(&DebugEvent, 1000)) {
        switch (DebugEvent.dwDebugEventCode) {
            
             case OUTPUT_DEBUG_STRING_EVENT:
                print("debug event OUTPUT_DEBUG_STRING_EVENT\n");
                break;
            case :
...
                break;
            default:
                print("unexpected debug event %d\n", DebugEvent.dwDebugEventCode);
                break;
        }
        if (DebugEvent.dwDebugEventCode == OUTPUT_DEBUG_STRING_EVENT) {
            if (ContinueDebugEvent(DebugEvent.dwProcessId, DebugEvent.dwThreadId, DBG_CONTINUE) == NULL) {
                print("fail ContinueDebugEvent\n");
            }
            break;
        }
        if (ContinueDebugEvent(DebugEvent.dwProcessId, DebugEvent.dwThreadId, DBG_EXCEPTION_NOT_HANDLED) == NULL) {
            print("fail ContinueDebugEvent\n");
        }
    }
    print("end of WaitForDebugEvent\n");
}

int
main(void)
{
    INIT();

    if (IsDebuggerPresent()) {
        debuggee();
        print("end of debuggee\n");
    }
    else {
        debugger();
        print("end of debugger\n");
    }

    return 0;
}

Basically it creates a new process with the same exe and the first process acts as a debugger on the second process (the debuggee) The debuggee calls OutputDebugString which results as a OUTPUT_DEBUG_STRING_EVENT in the WaitForDebugEvent loop. The debugger calls ContinueDebugEvent with DBG_CONTINUE to tell the kernel to let the debuggee continue. I do not know what is done in the kernel at that point.

The logs for the debuggee end with OutputDebugString making a sys call RaiseException

Entry into do_syscall to execute a non-ignorable system call
system call: sysnum = 0x0000012f, param_base = 0x0040f974
priv_mcontext_t @0x2040bf80
    xax = 0x0000012f
    xbx = 0x00000000
    xcx = 0x00000000
    xdx = 0x0040f974
    xsi = 0x00da9ce1
    xdi = 0x00000000
    xbp = 0x0040fc50
    xsp = 0x0040f970
    ymm0= 0x000000000000000000000000000000000825ff73ff734e004e000c251025ff73
    ymm1= 0x000000000000000000000000000000004e001c252025ff73ff734e004e002425
    ymm2= 0x0000000000000000000000000000000000000000000000000000000000000000
    ymm3= 0x0000000000000000000000000000000000000000000000000000000000000000
    ymm4= 0x0000000000000000000000000000000000000000000000000000000000000000
    ymm5= 0x0000000000000000000000000000000000000000000000000000000000000000
    ymm6= 0x0000000000000000000000000000000000000000000000000000000000000000
    ymm7= 0x000000000000000000000000000000000000000075636573797469726e69772d
    mxcsr=0x00001f80
    eflags = 0x00000246
    pc     = 0x76ec15ee
    param 0: 0x0040fc5c
    param 1: 0x0040f980
    param 2: 0x00000001
    param 3: 0x00010007
    param 4: 0x00497b60
    param 5: 0x00490000
    param 6: 0x00497fe0
    param 7: 0x00000001
    param 8: 0x0000003f
Call stack:
    0x76ec15ee [ntdll.dll~ZwRaiseException+0x12,~NtRaiseHardError-0x6] 
    frame ptr 0x0040fc50 => parent 0x0040fcac, 0x7515b727 [KERNELBASE.dll~RaiseException+0x58,~CloseHandle-0x9] 
    frame ptr 0x0040fcac => parent 0x0040ff18, 0x7516256b [KERNELBASE.dll~OutputDebugStringA+0x5b,~IsDebuggerPresent-0x299] 
    frame ptr 0x0040ff18 => parent 0x0040ff24, 0x00d8107e [security-win32.outputdebugstring.exe] 
    frame ptr 0x0040ff24 => parent 0x0040ff2c, 0x00d8129b [security-win32.outputdebugstring.exe] 
    frame ptr 0x0040ff2c => parent 0x0040ff74, 0x00d820b2 [security-win32.outputdebugstring.exe] 
    frame ptr 0x0040ff74 => parent 0x0040ff80, 0x74d133ca [KERNEL32.dll~BaseThreadInitThunk+0x12,~VerifyConsoleIoHandle-0xb3] 
    frame ptr 0x0040ff80 => parent 0x0040ffc0, 0x76ed9ed2 [ntdll.dll~RtlInitializeExceptionChain+0x63,~RtlAllocateActivationContextStack- 
    frame ptr 0x0040ffc0 => parent 0x0040ffd8, 0x76ed9ea5 [ntdll.dll~RtlInitializeExceptionChain+0x36,~RtlAllocateActivationContextStack- 
    frame ptr 0x0040ffd8 => parent 0x00000000, 0x00000000 [] 
fcache_enter = 0x204a2000, target = 0x204a2500

And we have nothing after that fcache_enter

I think that the big question is how the kernel returns to the debuggee. Any idea ?

[derekbruening]

This is a known problem. It has been in our documentation for years: http://dynamorio.org/docs/using.html#sec_using_debugger

It was in the pre-open-source issue tracker: looks like there's no entry in this tracker so this will serve.

Xref http://www.unixwiz.net/techtips/outputdebugstring.html

[Simorfo]

On Windows, if an application invokes OutputDebugString() while under a debugger, DynamoRIO can end up losing control of the application.

Ok, but do you know how to fix it ? Do you know how the kernel to gives back control to the debuggee ?

Here is one more information When I do GetThreadContext in the debugger on the debuggee, I get eip= 0x7515b727 which is the top return address from the call stack. If this is the case, maybe a solution is to push to the stack a return address to some dynamorio function like dispatch, what do you think ?

[derekbruening]

From old notes: "OutputDebugString() throws an exception that the debugger handles which means it sets the context back to some native PC and we lose control...It won't be hard to watch for this particular exception and modify the machine context, the only issue is that we're doing extra work which matters only when a debugger is present."

[derekbruening]

That last bit explains why we never got around to doing anything -- it seems worth fixing as we've long wanted to improve debugging (xref various issues on debugger integration: #532, https://github.com/DynamoRIO/drmemory/wiki/Projects#advanced-debugging-tools)