Open ionescu007 opened 8 years ago
Thank you for letting me know a way to "fix" the issue. I was unable to figure out that DriverVerifier was looking at PFN for the thread stack check. It is, personally, very interesting.
In addition to risk of 109 bug check you mentioned, adding manipulation of PFN may make code more fragile against updates in the Windows kernel, and may not help developers produce bug-free tools that run multiple platforms. Although I wish to run HyperPlatform with DriveVerifier for ease of debugging, I would think it is lower priority for this reason.
Thank you for providing very concrete information anyway. I would probably look into them for fun.
Note that this issue prevent us from using a checked build too. It would be worth considering fixing this issue to have better tools for testing.
Using KeExpandKernelStackAndCalloutEx() without returning from a callback, getting its stack base with IoGetInitialStack(), reusing it for a hypervisor as well as changing thread's stack info and/or PFN. That could potentially an option but still does seem to be a clean enough solution.
MmCreateKernelStack() could be handy for solving this issue if it were exported. https://bekirkarul.com/WRK/d3/d41/a02530_source.html#l03118
0: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
KERNEL_SECURITY_CHECK_FAILURE (139)
A kernel component has corrupted a critical data structure. The corruption
could potentially allow a malicious user to gain control of this machine.
Arguments:
Arg1: 0000000000000004, The thread's stack pointer was outside the legal stack
extents for the thread.
Arg2: ffffd0019a85c740, Address of the trap frame for the exception that caused the bugcheck
Arg3: ffffd0019a85c698, Address of the exception record for the exception that caused the bugcheck
Arg4: 0000000000000000, Reserved
Debugging Details:
------------------
(symbol related errors ommited)
DUMP_CLASS: 1
DUMP_QUALIFIER: 0
BUILD_VERSION_STRING: 10586.0.amd64chk.th2_release.151029-1700
DUMP_TYPE: 0
BUGCHECK_P1: 4
BUGCHECK_P2: ffffd0019a85c740
BUGCHECK_P3: ffffd0019a85c698
BUGCHECK_P4: 0
TRAP_FRAME: ffffd0019a85c740 -- (.trap 0xffffd0019a85c740)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=ffffd0019a85c900 rbx=0000000000000000 rcx=0000000000000004
rdx=ffffd0019a85c8a0 rsi=0000000000000000 rdi=0000000000000000
rip=fffff801ee5bf3fb rsp=ffffd0019a85c8d0 rbp=0000000000000001
r8=ffffe00163a7e040 r9=ffffd0019d788000 r10=ffffd0019d788000
r11=ffffd0019d78e000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up di pl zr na po nc
nt!RtlpGetStackLimits+0x27:
fffff801`ee5bf3fb int 29h
Resetting default scope
EXCEPTION_RECORD: ffffd0019a85c698 -- (.exr 0xffffd0019a85c698)
ExceptionAddress: fffff801ee5bf3fb (nt!RtlpGetStackLimits+0x0000000000000027)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 0000000000000004
Subcode: 0x4 FAST_FAIL_INCORRECT_STACK
CPU_COUNT: 2
CPU_MHZ: 8fc
CPU_VENDOR: GenuineIntel
CPU_FAMILY: 6
CPU_MODEL: 45
CPU_STEPPING: 1
CPU_MICROCODE: 0,0,0,0 (F,M,S,R) SIG: 1D'00000000 (cache) 0'00000000 (init)
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: 0x139
CURRENT_IRQL: 0
ANALYSIS_SESSION_HOST: DESKTOP-LQSEFPE
ANALYSIS_SESSION_TIME: 05-06-2016 06:04:20.0644
ANALYSIS_VERSION: 10.0.10586.567 amd64fre
LAST_CONTROL_TRANSFER: from fffff801ee4839d0 to fffff801ee679ac0
STACK_TEXT:
ffffd001`9a85bb68 fffff801`ee4839d0 : 00000000`00000000 00000000`00000139 00000000`00000003 00000000`00000065 : nt!DbgBreakPointWithStatus
ffffd001`9a85bb70 fffff801`ee484b2c : 00000000`00000003 ffffd001`00000000 fffff801`ee6839d0 00000000`00000139 : nt!KiBugCheckDebugBreak+0x14
ffffd001`9a85bbe0 fffff801`ee6742f4 : 00000000`00000139 00000000`00000004 ffffd001`9a85c740 ffffd001`9a85c698 : nt!KeBugCheck2+0xa38
ffffd001`9a85c3e0 fffff801`ee67efe9 : 00000000`00000139 00000000`00000004 ffffd001`9a85c740 ffffd001`9a85c698 : nt!KeBugCheckEx+0x104
ffffd001`9a85c420 fffff801`ee67f310 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiBugCheckDispatch+0x69
ffffd001`9a85c560 fffff801`ee67e4f3 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiFastFailDispatch+0xd0
ffffd001`9a85c740 fffff801`ee5bf3fb : ffffd001`9a85c8f0 ffffd001`9a85c930 ffffd001`9a85c938 00000000`00000000 : nt!KiRaiseSecurityCheckFailure+0xf3
ffffd001`9a85c8d0 fffff801`ee5bf419 : ffffd001`9a85c930 ffffd001`9a85c938 00000000`00000000 00000000`00000000 : nt!RtlpGetStackLimits+0x27
ffffd001`9a85c910 fffff801`ee5ccb95 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!RtlEnoughStackSpaceForStackCapture+0x15
ffffd001`9a85c950 fffff801`ee5cc159 : ffffe001`5ab56cc0 00000000`00000009 00000000`00000100 ffffd001`9a85cb38 : nt!RtlWalkFrameChain+0x6d
ffffd001`9a85c9a0 fffff801`ee4b39dc : 00000000`00000000 fffff6f0`00000008 ffffe001`5ab56cc0 ffffd001`9a85ca20 : nt!RtlCaptureStackBackTrace+0x55
ffffd001`9a85c9f0 fffff801`ee523ceb : 00000000`00000000 fffff6f0`00b26cb0 80000000`35794b63 80000000`35794963 : nt!MiSnapPte+0x80
ffffd001`9a85ca50 fffff801`ee4dfd39 : fffff6f0`00b26cb0 ffffd001`9a85cad0 00000000`00000000 00000000`00000000 : nt!MiSetNonPagedPoolNoSteal+0x53
ffffd001`9a85ca90 fffff801`ee4dfdd5 : ffffe001`64d96000 ffffd001`9a85cb38 ffffd001`9a85cb30 00000000`00000000 : nt!MiGetPhysicalAddress+0x199
ffffd001`9a85cb10 fffff801`315e43b4 : ffffe001`64d96000 fffff801`315e1c0b ffffe001`5ff8ed00 00000000`00000000 : nt!MmGetPhysicalAddress+0x19
ffffd001`9a85cb50 fffff801`315e1f36 : ffffe001`64d96000 00000000`00000000 00000000`00000000 00000000`00000000 : MemoryMon!UtilPaFromVa+0x14 [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\util.cpp @ 652]
ffffd001`9a85cb90 fffff801`315e1ecd : ffffe001`64d94000 00000000`00000002 00000000`fd4ec044 ffffe001`5ff8ed00 : MemoryMon!EptpConstructTables+0x236 [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\ept.cpp @ 334]
ffffd001`9a85cc30 fffff801`315e1e0d : ffffe001`63685000 00000000`00000003 00000000`fd4ec044 ffffe001`5ff8ed00 : MemoryMon!EptpConstructTables+0x1cd [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\ept.cpp @ 320]
ffffd001`9a85ccd0 fffff801`315e19b5 : ffffe001`63c4c000 ffffe001`00000004 00000000`fd4ec044 ffffe001`5ff8ed00 : MemoryMon!EptpConstructTables+0x10d [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\ept.cpp @ 304]
ffffd001`9a85cd70 fffff801`315e5bca : ffffe001`5ff8ed00 ffffe001`64398000 fffff801`315e3a90 fffff801`315e8320 : MemoryMon!EptHandleEptViolation+0x115 [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\ept.cpp @ 454]
ffffd001`9a85ce00 fffff801`315e6e27 : ffffd001`9a85cf18 ffffe001`64398000 fffff801`315e3a90 fffff801`315e7f90 : MemoryMon!VmmpHandleEptViolation+0x4a [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\vmm.cpp @ 949]
ffffd001`9a85ce60 fffff801`315e4dbb : ffffd001`9a85cf18 00000000`00000000 00000000`00000000 00000000`00000000 : MemoryMon!VmmpHandleVmExit+0x247 [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\vmm.cpp @ 263]
ffffd001`9a85cee0 fffff801`315e1345 : ffffd001`9a85cf70 00000000`00000000 00000000`00000000 00000000`00000000 : MemoryMon!VmmVmExitHandler+0xcb [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\vmm.cpp @ 185]
ffffd001`9a85cf50 ffffd001`9a85cf70 : 00000000`00000000 00000000`00000000 00000000`00000000 fffff801`ee72aca0 : MemoryMon!AsmVmmEntryPoint+0x25 [C:\Users\user\Desktop\HP\MemoryMon\HyperPlatform\HyperPlatform\Arch\x64\x64.asm @ 147]
ffffd001`9a85cf58 00000000`00000000 : 00000000`00000000 00000000`00000000 fffff801`ee72aca0 fffff801`ee727180 : 0xffffd001`9a85cf70
STACK_COMMAND: kb
THREAD_SHA1_HASH_MOD_FUNC: aeaa00146335439d3364d4e0241570500ff6118b
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: faf49cab4bf906c5287392055c32b88264be6f59
THREAD_SHA1_HASH_MOD: a9a47b2edfafd65f06ae969eca355f3076a1b1ef
FOLLOWUP_IP:
MemoryMon!UtilPaFromVa+14 [c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\util.cpp @ 652]
fffff801`315e43b4 mov qword ptr [rsp+20h],rax
FAULT_INSTR_CODE: 24448948
FAULTING_SOURCE_LINE: c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\util.cpp
FAULTING_SOURCE_FILE: c:\users\user\desktop\hp\memorymon\hyperplatform\hyperplatform\util.cpp
FAULTING_SOURCE_LINE_NUMBER: 652
FAULTING_SOURCE_CODE:
648: }
649:
650: // VA -> PA
651: _Use_decl_annotations_ ULONG64 UtilPaFromVa(void *va) {
> 652: const auto pa = MmGetPhysicalAddress(va);
653: return pa.QuadPart;
654: }
655:
656: // VA -> PFN
657: _Use_decl_annotations_ PFN_NUMBER UtilPfnFromVa(void *va) {
SYMBOL_STACK_INDEX: f
SYMBOL_NAME: MemoryMon!UtilPaFromVa+14
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: MemoryMon
IMAGE_NAME: MemoryMon.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 572c9284
BUCKET_ID_FUNC_OFFSET: 14
FAILURE_BUCKET_ID: 0x139_4_MemoryMon!UtilPaFromVa
BUCKET_ID: 0x139_4_MemoryMon!UtilPaFromVa
PRIMARY_PROBLEM_CLASS: 0x139_4_MemoryMon!UtilPaFromVa
TARGET_TIME: 2016-05-06T12:57:39.000Z
OSBUILD: 10586
OSSERVICEPACK: 0
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 272
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2015-10-29 19:53:36
BUILDDATESTAMP_STR: 151029-1700
BUILDLAB_STR: th2_release
BUILDOSVER_STR: 10.0.10586.0.amd64chk.th2_release.151029-1700
ANALYSIS_SESSION_ELAPSED_TIME: 321c
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:0x139_4_memorymon!utilpafromva
FAILURE_ID_HASH: {63c9aa7c-5f0c-b05e-2dc5-bf2d1b41f756}
Followup: MachineOwner
---------
Hi,
Just to let you know, the correct way to "fix" this issue is to edit the PFNs for the kernel stack, and to set them as kernel stack pages (one of the flags in e2.u1 I believe) as well as write the KTHREAD as the u1.KernelStackOwner, and then OR'ing with 1. You can see this by doing a !pte on a kernel stack page (a real one), and then !pfn, and analyzing the MMPFN structure.
This requires pretty low-level hacking of the PFN database, but in case you were curious, that's why Verifier/Kernel don't like it when you do API calls from VMM context. It's also technically possible to get 109 bug check in this situation.
You may want to investigate KeExpandKernelStackAndCalloutEx as a possible way to avoid this -- it should create a 'legitimate' stack for the VMM.