Proctrace failed to trace console mode pe file

[zdzhjx]

Windows GUI file works well, but failed with console mode file. I have tested it in two ways: 1) Directly run from cmd.exe in windows 7, can not see any trace message. 2) Run from drakvuf with -e and -i option, with -i option set value to pid of explorer.exe, some time it can work, but at most time it can not work.

[v-p-b]

This is a known issue. Console applications run through conhost.exe since Windows 7 (to thwart shatter attacks) that uses some weird method to spawn the real process. I've been trying to figure out how we could catch process startup =and find the moduels in memory) with no success yet.

[tklengyel]

IMHO catching process startup would be easiest through CR3 events. By maintaining a list of active processes we could easily detect if the CR3 that is being scheduled new or not. Of course we should also catch the event when a process exits to remove it from the active list, as after that it would be possible that the CR3 value gets recycled.

[zdzhjx]

I have tried CR3 event, but it is diffcult to get the right time to inject trap. I mean to trap api function in kernel32.dll.

[v-p-b]

As @zdzhjx said it's unfortunately not trivial to tell when a process is in a "trapable" state: I just found that in case of console applications (launched through conhost) even at the time the CR3 is assigned and module information is populated in PEB the modules aren't actually present in memory yet! In case of GUI apps the delay between the loading of the main module and its dependencies also caused some trouble but I managed to resolve that by placing traps at a later point in time (on calling PsGetCurrentThreadTeb). Unfortunately in case of console apps this API is only called at conhost startup, but not at the loading of the real executable.

[zdzhjx]

I had thought a way: 1) start the target app with SUSPEND tag 2) create a new thread, in the thread, sleep some time, then add traps into the app's EOP, resume the thread 3) add other traps EOP'trap invoked;

I have not test it, do you think is it possible?

[v-p-b]

@zdzhjx Is this supposed to be implemented in the injector?

[zdzhjx]

@v-p-b ， yes, need to change injector

[v-p-b]

This may be a workaround, but the problem should be solvable at the plugins level. I found that I can't (trivially) make SYSCALLS trap on loader APIs (Ldr*), which is strange (any ideas about this @tklengyel ?) but these interfaces may lead to a solution: http://www.nynaeve.net/?p=205

[tklengyel]

@buherator: you mean trapped Ldr* functions in the kernel don't actually get a callback when you would expect them to?

[v-p-b]

@tklengyel Exactly, yes. I did experiments with hooking only APIs starting with "Ld" and also getting rid of the corresponding filter branch in SYSCALLS but got no callback hits at all.

[tklengyel]

@v-p-b: trapping Ldr functions in the kernel works fine:

DRAKVUF v0.3-284a71a [SYSCALL] vCPU:1 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrResSearchResource [SYSCALL] vCPU:1 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrpResGetMappingSize [SYSCALL] vCPU:1 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrpResSearchResourceMappedFile [SYSCALL] vCPU:1 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrpResGetResourceDirectory [SYSCALL] vCPU:0 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrpResSearchResourceInsideDirectory [SYSCALL] vCPU:0 CR3:0x6f92d000,svchost.exe ntoskrnl.exe!LdrpResCompareResourceNames ...

However, the Ldr functions that article you linked earlier don't reside in the kernel like LdrInitializeThunk, those are ntdll functions living in userspace. So those can get easily relocated/paged out. What would be the best is if we could identify the kernel-side of those callbacks and trap there, just before the callback switches to userspace.

[v-p-b]

@tklengyel Yes, but neither could I get SYSCALLS to produce the output like the one you just posted, which indicates that the problem is with my setup. Hopefully I'll get the time to do some more debugging tonight. Thanks!

[zdzhjx]

@tklengyel @v-p-b , It seems that drakvuf_get_current_thread_id can not get thread id , when it running in callback of userspace traps. While it works fine in kernel space, any idea on how to resolve it? Thanks.

[tklengyel]

Would need more information about why it doesn't work. Does fs/gs no longer point to the kpcr or something else?

[v-p-b]

So it seems that I'm unable to do v2p translation for some reason: https://github.com/v-p-b/drakvuf/blob/proctracer_rebase/src/plugins/proctracer/proctracer.cpp#L206

vmi_pagetable_lookup() comes back with 0, didn't have luck with vmi_pagetable_lookup_extended() either...

EDIT: v2p only fails in case of certain console binaries... ftp.exe for example seems to work fine o.O

[tklengyel]

I think this came up before by another user too, the VA doesn't seem to be mapped into the process yet if the process is still just starting. There is no easy solution to this other then creating memaccess write watches on the the process' pagetables to catch when an entry gets written in there until the VA shows up.. On Jun 11, 2016 07:38, "buherator" notifications@github.com wrote:

So it seems that I'm unable to do v2p translation for some reason: https://github.com/v-p-b/drakvuf/blob/proctracer_rebase/src/plugins/proctracer/proctracer.cpp#L206

vmi_pagetable_lookup() comes back with 0, didn't have luck with vmi_pagetable_lookup_extended() either...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tklengyel/drakvuf/issues/162#issuecomment-225362278, or mute the thread https://github.com/notifications/unsubscribe/ADBp3YUlggf6-OUQ7qAQ7Q7uteHk2Irsks5qKro5gaJpZM4IqQy4 .

[v-p-b]

Can you show me an example of how to create such a watch?

[tklengyel]

There isn't any available right now. You can use vmi_get_va_pages to loop through all entries in the process' tables. Then you want to add post-type trap of every location thats in there (including where the cr3 points to). When you get a write access, you should check if the va has showed up. If not, do another get_va_pages and add a watch to the entry that may have been added just now. You keep doing that until the va shows up. It's a pretty slow approach but it should work. On Jun 11, 2016 08:11, "buherator" notifications@github.com wrote:

Can you show me an example of how to create such a watch?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tklengyel/drakvuf/issues/162#issuecomment-225364504, or mute the thread https://github.com/notifications/unsubscribe/ADBp3R3tJASrM7zMWPLMIt5YZ7XWDFHYks5qKsIGgaJpZM4IqQy4 .

[v-p-b]

Thanks, I'll look these up and think through how I can move forward

[v-p-b]

I did a quick loop over the list returned by vmi_get_va_pages and it seems that the VA is present at the time I want to place the userland traps:

GSList *pages=vmi_get_va_pages(vmi,cr3);
while(pages){
    page_info_t *pi=(page_info_t*)pages->data;
    if (pi->vaddr<=dllbase && dllbase<pi->vaddr+pi->size) printf("Page found!\n");
    pages=pages->next;
}  

Result (with debug printfs):

  10 PID: 2648
  11 Module name: pdf.exe @ 840000
  12 Page found!
  13 Adding trap for CR3:b9ee9540 @ 0 (840000+10ca0): f000ff53

The "@ 0" in line 13 shows that v2p returned 0. Line 12 shows that the VA of the module base (840000) was present in the vmi_get_va_pages list.

[tklengyel]

Well, you are checking for dllbase only. That is a different page then dllbase + 10ca0. Remember, page size is 4096, so if dllbase is mapped you only have dllbase + 4095 there.

[zdzhjx]

@tklengyel while debuging function drakvuf_get_current_thread_id , I got some output here:

addr of thread=7ffdf124 // the location va where to got thread fsgs=0x7ffdf000 // got from vmi_get_vcpureg prcb=0x120 // drakvuf->offsets[KPCR_PRCBDATA] offset=0x4 // drakvuf->offsets[KPRCB_CURRENTTHREAD] reg_fsgs=0 // not set to regs->gs_base vcpuid=0 // only one vcpu cr3=0x61dcf000 // correct cr3, current process

these will get thread=0

and I using following code will work, but it must have pid:

addr_t get_tid( vmi_instance_t vmi, uint64_t vcpu_id, vmi_pid_t pid ) { addr_t thread; addr_t prcb; reg_t fsgs; vmi_get_vcpureg(vmi, &fsgs, FS_BASE, vcpu_id);

if (VMI_SUCCESS != vmi_read_addr_va(vmi, fsgs + 0x24, pid, &thread)) // for 32 bit os only
{
    printf("** pid=%d fsgs=%lx\n", pid, fsgs);
    return 0;
}
return thread;

}

any idea?