search

DynamoRIO / dynamorio

Dynamic Instrumentation Tool Platform
Other
2.61k stars 554 forks source link

ASSERT incorrect syscall numbers on Windows 10.0.15063.1387 due to Crowdstrike Falcon #3243

Open fmoessbauer opened 5 years ago

fmoessbauer commented 5 years ago

On this build version, no applications can be run under DynamoRIO. No logs were generated.

$ DynamoRIO-Windows-7.0.17832-0\bin64\drrun.exe -debug -verbose -loglevel 3 -- notepad.exe
INFO: targeting application: "C:\WINDOWS\system32\notepad.exe"
INFO: app cmdline:  "notepad.exe"
INFO: configuration directory is "C:\Users\felix/dynamorio"
INFO: created child with pid 11728 for C:\WINDOWS\system32\notepad.exe
INFO: waiting forever for app to exit...
<WARNING: Running on unsupported Windows 10+ version>
<Application C:\WINDOWS\system32\notepad.exe (11728).  Internal Error: DynamoRIO debug check failure: ..\..\core\win32\ntdll.c:623 (byte *)get_proc_address(ntdllh, syscall_names[i]) != NULL && (*((int *)(((byte *)get_proc_address(ntdllh, syscall_names[i])) + SYSNUM_OFFS)) == syscalls[i] || ALLOW_HOOKER((byte *)get_proc_address(ntdllh, syscall_names[i])) || (i == SYS_TestAlert && *(uint *)((byte *)get_proc_address(ntdllh, syscall_names[i])) == 0xe9505050))
(Error occurred @-1 frags)
version 7.0.17832, custom build
>

With previous Windows build versions, DynamoRIO worked as expected.

derekbruening commented 5 years ago

Why didn't the code right before that assert succeed at finding the syscall numbers? Do you have invasive security software on your machine that injects and hooks some of these syscalls? I would suggest looking at exactly which syscall this is (syscall_names[i]) and disassembling that syscall wrapper in ntdll to see what's going on: is it hooked on your machine?

fmoessbauer commented 5 years ago

How can I see if it is hooked? Does DR print a message in this case? The related syscall is NtAllocateVirtualMemoryEx at i=16:

[16] @ 00000000`1551e220 0n116
[16] @ 00000000`154f2fe0 0x00000000`154a9ac8  "NtAllocateVirtualMemoryEx"

It seems that this function is not exported by ntdll.dll:

$ dumpbin -EXPORTS ntdll.dll | grep AllocateVirtualMemory
        213   CC 000A55F0 NtAllocateVirtualMemory
       1682  689 000A55F0 ZwAllocateVirtualMemory
derekbruening commented 5 years ago

Disassemble it to see if there's anything unusual preventing DR from finding the number, like a hook.

That is strange that it would not be exported as these ntdll wrappers have always been in the past, and the syscall is still present according to https://github.com/DynamoRIO/dynamorio/pull/3206/files#diff-b224ac300329530c2ca771e247d074fa . Does the DR get_proc_address call fail?

fmoessbauer commented 5 years ago

Hi, this seems to be not related to windows updates but to Crowdstrike Falcon. The system I'm working on is a companies machine, so I was not aware of that (it was installed without notice ...). IMO you can close the issue as it's not the fault of DR. Anyways: Thanks for your assistance.

For the sake of completeness here is the disassembly around address 000A55F0:

200034   00000001800A55E2: 0F 05              syscall
200035   00000001800A55E4: C3                 ret
200036   00000001800A55E5: CD 2E              int         2Eh
200037   00000001800A55E7: C3                 ret
200038   00000001800A55E8: 0F 1F 84 00 00 00  nop         dword ptr [rax+rax+0000000000000000h]
200039                     00 00
200040   00000001800A55F0: 4C 8B D1           mov         r10,rcx
200041   00000001800A55F3: B8 18 00 00 00     mov         eax,18h
200042   00000001800A55F8: F6 04 25 08 03 FE  test        byte ptr [000000007FFE0308h],1
200043                     7F 01
200044   00000001800A5600: 75 03              jne         00000001800A5605
200045   00000001800A5602: 0F 05              syscall
200046   00000001800A5604: C3                 ret
200047   00000001800A5605: CD 2E              int         2Eh
200048   00000001800A5607: C3                 ret
200049   00000001800A5608: 0F 1F 84 00 00 00  nop         dword ptr [rax+rax+0000000000000000h]
200050                     00 00
200051   00000001800A5610: 4C 8B D1           mov         r10,rcx
200052   00000001800A5613: B8 19 00 00 00     mov         eax,19h
200053   00000001800A5618: F6 04 25 08 03 FE  test        byte ptr [000000007FFE0308h],1
200054                     7F 01
200055   00000001800A5620: 75 03              jne         00000001800A5625
200056   00000001800A5622: 0F 05              syscall
200057   00000001800A5624: C3                 ret
200058   00000001800A5625: CD 2E              int         2Eh
200059   00000001800A5627: C3                 ret
derekbruening commented 5 years ago

Could you elaborate on how Crowdstrike Falcon caused problems? Your disassembly above looks normal for Win10 1511+, and:

Strike what I said about expecting NtAllocateVirtualMemoryEx to be exported: I assumed this was a more recent Win10, based on the output in your first post <WARNING: Running on unsupported Windows 10+ version>. Yet 10.0.15063.* should be Win10 1703 (right?), so we do not expect NtAllocateVirtualMemoryEx to be present. If DR got the version right, it would have hit the continue case in the loop and would not hit this assert for that syscall. So this seems to be a problem with DR deducing the wrong version and not figuring out that this is 1703.

The code that does this is at https://github.com/DynamoRIO/dynamorio/blob/master/core/win32/os.c#L643 Could you look at your ntdll wrapper for NtGetContextThread? The one you pasted above for NtAllocateVirtualMemory looks normal and looks like DR should get the number, so if NtGetContextThread has a hook or is weird it could explain it. Otherwise if you could step through or printf-debug that section of code to see why it ends up with syscalls == NULL.

fmoessbauer commented 5 years ago

I just time-correlated what has changed on my system since DR was last known to be working. I found this more or less related issue. According to the last-modified timestamp and checksum of ntdll, CrowdStrike did not modify the dll (at least not on disk). After fully removing CrowdStrike Falcon and a reboot, DR worked correctly again (and no "unsupported windows 10+" warning is shown anymore).

The windows version (according to winver) is 10.0.15063.1387 build 1703 (as you guessed right). Unfortunately I cannot do further analysis as CrowdStrike Falcon is now removed. However the checksum of ntdll has not changed after deinstalling.

derekbruening commented 5 years ago

The theory is that Crowdstrike Falcon hooked NtGetContextThread.

fmoessbauer commented 5 years ago

Crowdstrike Falcon was installed again, so I can now further debug the issue:

The OS version was then detected as os_version = WINDOWS_VERSION_10_1703; in L604 In L649 everything is strange. But this might be as I use a optimized DR version (as I was not able to build DR...):

num_GetContextThread 0n691179280 <-- not sure if correct
     (0n424086 according to .call syscalls_init_get_num(ntdllh, SYS_GetContextThread))
num_AllocateVirtualMemory 0n0 <-- not sure if correct
     (0n430754 according to .call syscalls_init_get_num(ntdllh, SYS_AllocateVirtualMemory))
SYS_GetContextThread 0n3
SYS_AllocateVirtualMemory 0n15
syscalls[3] 0n233
syscalls[15] 0n24

Disassembly at ~L649 

00000000`15357d34 83ffff          cmp     edi,0FFFFFFFFh
00000000`15357d37 7406            je      dynamorio!windows_version_init+0x3af (00000000`15357d3f)
00000000`15357d39 413b780c        cmp     edi,dword ptr [r8+0Ch]
00000000`15357d3d 7513            jne     dynamorio!windows_version_init+0x3c2 (00000000`15357d52)
00000000`15357d3f 83feff          cmp     esi,0FFFFFFFFh
00000000`15357d42 0f8494000000    je      dynamorio!windows_version_init+0x44c (00000000`15357ddc)
00000000`15357d48 413b703c        cmp     esi,dword ptr [r8+3Ch]
00000000`15357d4c 0f848a000000    je      dynamorio!windows_version_init+0x44c (00000000`15357ddc)

Where
edi=67cba
esi=696c6

The disassembly around address A7000 in ntdll.dll:

 00000001800A6FF4: C3                 ret
  00000001800A6FF5: CD 2E              int         2Eh
  00000001800A6FF7: C3                 ret
  00000001800A6FF8: 0F 1F 84 00 00 00  nop         dword ptr [rax+rax+0000000000000000h]
                    00 00
  00000001800A7000: 4C 8B D1           mov         r10,rcx
  00000001800A7003: B8 E9 00 00 00     mov         eax,0E9h
  00000001800A7008: F6 04 25 08 03 FE  test        byte ptr [000000007FFE0308h],1
                    7F 01
  00000001800A7010: 75 03              jne         00000001800A7015
  00000001800A7012: 0F 05              syscall
  00000001800A7014: C3                 ret
  00000001800A7015: CD 2E              int         2Eh
  00000001800A7017: C3                 ret
  00000001800A7018: 0F 1F 84 00 00 00  nop         dword ptr [rax+rax+0000000000000000h]
                    00 00
  00000001800A7020: 4C 8B D1           mov         r10,rcx
  00000001800A7023: B8 EA 00 00 00     mov         eax,0EAh
  00000001800A7028: F6 04 25 08 03 FE  test        byte ptr [000000007FFE0308h],1
                    7F 01
  00000001800A7030: 75 03              jne         00000001800A7035
  00000001800A7032: 0F 05              syscall
  00000001800A7034: C3                 ret
  00000001800A7035: CD 2E              int         2Eh
  00000001800A7037: C3                 ret
  00000001800A7038: 0F 1F 84 00 00 00  nop         dword ptr [rax+rax+0000000000000000h]
                    00 00
  00000001800A7040: 4C 8B D1           mov         r10,rcx
  00000001800A7043: B8 EB 00 00 00     mov         eax,0EBh
  00000001800A7048: F6 04 25 08 03 FE  test        byte ptr [000000007FFE0308h],1
                    7F 01
  00000001800A7050: 75 03              jne         00000001800A7055
  00000001800A7052: 0F 05              syscall
derekbruening commented 5 years ago

num_GetContextThread 0n691179280 <-- not sure if correct (0n424086 according to .call syscalls_init_get_num(ntdllh, SYS_GetContextThread)) syscalls[3] 0n233

The 0n233 is correct: the first two are not.

num_AllocateVirtualMemory 0n0 <-- not sure if correct (0n430754 according to .call syscalls_init_get_num(ntdllh, SYS_AllocateVirtualMemory)) syscalls[15] 0n24

Again, 0n24 (0xe9 as can be seen at 0x1800A7003) is correct.

This is weird: any idea where those unusual values came from? Step through syscalls_init_get_num()?

fmoessbauer commented 5 years ago

Both calls to syscalls_init_get_num() return via L382 in ntdll.c (dereferencing the pointer). The local vars in this function are :

call 1

ntdll: 0x00007ffa`5dfa0000
sys_enum: 0n3
wrapper: 0x00007ffa`5e047000
$retreg=000000000006780a

call 2

ntdll: 0x00007ffa`5dfa0000
sys_enum: 0n15
wrapper:  0x00007ffa`5e0455f0
$retreg=0000000000069216

Finally windows_version_init is called with parameters:

# first debugging run
1:003> r ecx
rcx=6780a
1:003> r edx
rdx=69216
# second debugging run
1:004> r ecx
ecx=67a32
1:004> r edx
edx=6943e

This looks fishy to me. I guess that the issue is in dereferencing the pointer in L382. This value changes with each debugging run.