search

tandasat / HyperPlatform

Intel VT-x based hypervisor aiming to provide a thin VM-exit filtering platform on Windows.
MIT License
1.53k stars 408 forks source link

Kaspersky Internet Security conflict #63

Closed YangKi1902 closed 5 years ago

YangKi1902 commented 5 years ago

Hello, i got BSOD after install Kaspersky Internet Security, tested with 3 computer windows 10 64 bit, in Hyperplatform.log seem like nothing weird, here my crash dump log :

MANUALLY_INITIATED_CRASH (e2) The user manually initiated this crash dump. Arguments: Arg1: 0000000000000001 Arg2: ffffe5016aef7d28 Arg3: fffff8048398136f Arg4: 0000000000000000

Debugging Details:

KEY_VALUES_STRING: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 401

BUILD_VERSION_STRING: 17763.1.amd64fre.rs5_release.180914-1434

SYSTEM_MANUFACTURER: Micro-Star International Co., Ltd.

SYSTEM_PRODUCT_NAME: GT72 2QD

SYSTEM_SKU: To be filled by O.E.M.

SYSTEM_VERSION: REV:0.C

BIOS_VENDOR: American Megatrends Inc.

BIOS_VERSION: E1781IMS.316

BIOS_DATE: 09/23/2015

BASEBOARD_MANUFACTURER: Micro-Star International Co., Ltd.

BASEBOARD_PRODUCT: MS-1781

BASEBOARD_VERSION: REV:0.C

DUMP_TYPE: 1

BUGCHECK_P1: 1

BUGCHECK_P2: ffffe5016aef7d28

BUGCHECK_P3: fffff8048398136f

BUGCHECK_P4: 0

CPU_COUNT: 8

CPU_MHZ: a86

CPU_VENDOR: GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 47

CPU_STEPPING: 1

CPU_MICROCODE: 6,47,1,0 (F,M,S,R) SIG: 1D'00000000 (cache) 1D'00000000 (init)

BLACKBOXBSD: 1 (!blackboxbsd)

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

BUGCHECK_STR: 0xE2

PROCESS_NAME: System

CURRENT_IRQL: 2

ANALYSIS_SESSION_HOST: DESKTOP-6P18NJQ

ANALYSIS_SESSION_TIME: 05-01-2019 05:36:30.0374

ANALYSIS_VERSION: 10.0.17763.132 amd64fre

BAD_STACK_POINTER: ffffe5016aef7c18

LAST_CONTROL_TRANSFER: from fffff804864d74fe to fffff8047cdb6730

STACK_TEXT:
ffffe5016aef7c18 fffff804864d74fe : 00000000000000e2 0000000000000001 ffffe5016aef7d28 fffff8048398136f : nt!KeBugCheckEx ffffe5016aef7c20 fffff804864d7b03 : ffffe5016aef7d28 ffffe50158307000 fffff804864d3f30 fffff804864d8840 : Hyperplatform!VmmpHandleUnexpectedExit+0x5e [e:\source\hyperplatform2\hyperplatform\vmm.cpp @ 348] ffffe5016aef7c70 fffff804864d4b0e : ffffe5016aef7d28 0000000000000000 0000000000000000 0000000000000000 : Hyperplatform!VmmpHandleVmExit+0x2b3 [e:\source\hyperplatform2\hyperplatform\vmm.cpp @ 329] ffffe5016aef7cf0 fffff804864d1470 : ffffe5016aef7de8 0000000000000000 0000000000000000 0000000000000000 : Hyperplatform!VmmVmExitHandler+0xde [e:\source\hyperplatform2\hyperplatform\vmm.cpp @ 212] ffffe5016aef7d60 fffff80483981372 : fffff80483981403 0000000000000000 ffffe5015f8a6330 fffff8047bae0180 : Hyperplatform!AsmVmmEntryPoint+0x4d [E:\Source\hyperplatform2\HyperPlatform\Arch\x64\x64.asm @ 191] fffff8047f5d6768 fffff80483981403 : 0000000000000000 ffffe5015f8a6330 fffff8047bae0180 ffffe5015f8a6240 : intelppm!MWaitIdle+0x22 fffff8047f5d6770 fffff8047cce9bbb : 0000000000000000 0000000000000081 0000000000000000 00000000000000ba : intelppm!AcpiCStateIdleExecute+0x23 fffff8047f5d67a0 fffff8047cce936f : 0000000000000000 0000000000000002 0000000000000002 0000000000000008 : nt!PpmIdleExecuteTransition+0x6bb fffff8047f5d6ac0 fffff8047cdba11c : 0000000000000000 fffff8047bae0180 fffff8047d162400 ffffe5015899b080 : nt!PoIdle+0x33f fffff8047f5d6c20 0000000000000000 : fffff8047f5d7000 fffff8047f5d0000 0000000000000000 0000000000000000 : nt!KiIdleLoop+0x2c

THREAD_SHA1_HASH_MOD_FUNC: 6c78bd86ecdabf246e57e5a72c011126206430a4

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: e9ae3369e7b00bc2a7eadd7b71ef2d56b67749fa

THREAD_SHA1_HASH_MOD: 5f52eff5d061dc0c69fdc7efcfc2896a9ad948b4

FOLLOWUP_IP: Hyperplatform!VmmpHandleUnexpectedExit+5e [e:\source\hyperplatform2\hyperplatform\vmm.cpp @ 348] fffff804`864d74fe 4883c448 add rsp,48h

FAULT_INSTR_CODE: 48c48348

FAULTING_SOURCE_LINE: e:\source\hyperplatform2\hyperplatform\vmm.cpp

FAULTING_SOURCE_FILE: e:\source\hyperplatform2\hyperplatform\vmm.cpp

FAULTING_SOURCE_LINE_NUMBER: 348

FAULTING_SOURCE_CODE:
344: const auto qualification = UtilVmRead(VmcsField::kExitQualification); 345: HYPERPLATFORM_COMMON_BUG_CHECK(HyperPlatformBugCheck::kUnexpectedVmExit, 346: reinterpret_cast(guest_context), 347: guest_context->ip, qualification);

348: } 349: 350: // MTF VM-exit 351: _Use_declannotations static void VmmpHandleMonitorTrap( 352: GuestContext *guest_context) { 353: VmmpDumpGuestState();

SYMBOL_STACK_INDEX: 1

SYMBOL_NAME: Hyperplatform!VmmpHandleUnexpectedExit+5e

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Hyperplatform

IMAGE_NAME: Hyperplatform.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 5cc8bd36

STACK_COMMAND: .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET: 5e

FAILURE_BUCKET_ID: 0xE2_STACKPTR_ERROR_Hyperplatform!VmmpHandleUnexpectedExit

BUCKET_ID: 0xE2_STACKPTR_ERROR_Hyperplatform!VmmpHandleUnexpectedExit

PRIMARY_PROBLEM_CLASS: 0xE2_STACKPTR_ERROR_Hyperplatform!VmmpHandleUnexpectedExit

TARGET_TIME: 2019-04-30T22:34:46.000Z

OSBUILD: 17763

OSSERVICEPACK: 0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 272

PRODUCT_TYPE: 1

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS

OS_LOCALE:

USER_LCID: 0

OSBUILD_TIMESTAMP: 2005-12-02 14:58:59

BUILDDATESTAMP_STR: 180914-1434

BUILDLAB_STR: rs5_release

BUILDOSVER_STR: 10.0.17763.1.amd64fre.rs5_release.180914-1434

ANALYSIS_SESSION_ELAPSED_TIME: 23ed

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:0xe2_stackptr_error_hyperplatform!vmmphandleunexpectedexit

FAILURE_ID_HASH: {f7d4f0ac-b587-5ae3-88d9-ad99782b047a}

Followup: MachineOwner

tandasat commented 5 years ago

Hi, can you please take and upload the kernel memory dump? I am specifically interested in what the instruction at the guest RIP and what is the VM-exit reason. My guess is that the anti-virus has a hypervisor with nesting support but nesting is broken somehow, but I am unsure. Thank you.

YangKi1902 commented 5 years ago

hello, here is my complete memory.dmp and Hyperplatform.log : https://drive.google.com/file/d/1VY2NMU7kRiksojIvUlA6LeGzuvyNNvdd/view?usp=sharing

if you have any quick fix please tell me know, this is the first time im trying to implement Hyperplatform to my anticheat project and it will online today with about 300 users for test :D, Hyperplatform is a very good project that helped me so much about Patchguard problem, thank you.

tandasat commented 5 years ago

Thank you for sharing the dump. Can you also attach the PDB file of the build please?

YangKi1902 commented 5 years ago

sure, https://drive.google.com/file/d/1PNEsI-gfHKvlxabokUcJcdtZrVLrAwIl/view?usp=sharing.

tandasat commented 5 years ago

Thanks. It is interesting that the exit reason is 3: INIT signal. An INIT signal arrived which I have never had to deal with. Can you set true to kVmmpEnableRecordVmExit and test it on a single processor system? That would record vm-exits HyperPlatform handled and help me to trace how the guest reached to the state.

Also, is it possible to repro the issue in a VMware VM? Is there a free trial edition I can use for repro? I do not think I am going to have time to look into deeper anytime soon, but am curious what's happening.

YangKi1902 commented 5 years ago

sure, here new memory dump and log https://drive.google.com/file/d/1VY2NMU7kRiksojIvUlA6LeGzuvyNNvdd/view?usp=sharing

im just test with my real pc, but i think yes you can install the trial version of Kaspersky Internet Security and test it in VMware VM.

YangKi1902 commented 5 years ago

im not tested my self but some my users reported that BSOD with ESET Antivirus too, here the log :

Microsoft (R) Windows Debugger Version 10.0.17763.132 AMD64 Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Users\MyNameIs\Desktop\New folder (2)\New folder\050119-8392-01.dmp] Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: srv* Executable search path is: Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x64 Product: WinNt, suite: TerminalServer SingleUserTS Built by: 7601.24260.amd64fre.win7sp1_ldr.180908-0600 Machine Name: Kernel base = 0xfffff80002e02000 PsLoadedModuleList = 0xfffff8000303cc90 Debug session time: Wed May 1 17:27:36.308 2019 (UTC + 7:00) System Uptime: 0 days 5:41:09.791 Loading Kernel Symbols ............................................................... ................................................................ ......................... Loading User Symbols Loading unloaded module list .......

Use !analyze -v to get detailed debugging information.

BugCheck A, {fffff88008e41000, 0, 1, fffff80002fa5bb3}

Probably caused by : memory_corruption ( nt!MiGetWorkingSetInfo+263 )

Followup: MachineOwner

3: kd> !analyze -v

IRQL_NOT_LESS_OR_EQUAL (a) An attempt was made to access a pageable (or completely invalid) address at an interrupt request level (IRQL) that is too high. This is usually caused by drivers using improper addresses. If a kernel debugger is available get the stack backtrace. Arguments: Arg1: fffff88008e41000, memory referenced Arg2: 0000000000000000, IRQL Arg3: 0000000000000001, bitfield : bit 0 : value 0 = read operation, 1 = write operation bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status) Arg4: fffff80002fa5bb3, address which referenced memory

Debugging Details:

KEY_VALUES_STRING: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 400

BUILD_VERSION_STRING: 7601.24260.amd64fre.win7sp1_ldr.180908-0600

SYSTEM_MANUFACTURER: System manufacturer

SYSTEM_PRODUCT_NAME: System Product Name

SYSTEM_SKU: To Be Filled By O.E.M.

SYSTEM_VERSION: System Version

BIOS_VENDOR: American Megatrends Inc.

BIOS_VERSION: 0402

BIOS_DATE: 10/29/2010

BASEBOARD_MANUFACTURER: ASUSTeK Computer INC.

BASEBOARD_PRODUCT: P7H55-M LX

BASEBOARD_VERSION: X.0x

DUMP_TYPE: 2

BUGCHECK_P1: fffff88008e41000

BUGCHECK_P2: 0

BUGCHECK_P3: 1

BUGCHECK_P4: fffff80002fa5bb3

WRITE_ADDRESS: GetPointerFromAddress: unable to read from fffff800030a0100 Unable to get MmSystemRangeStart GetUlongPtrFromAddress: unable to read from fffff800030a02f0 GetUlongPtrFromAddress: unable to read from fffff800030a04a8 GetPointerFromAddress: unable to read from fffff800030a00d8 fffff88008e41000

CURRENT_IRQL: 0

FAULTING_IP: nt!MiGetWorkingSetInfo+263 fffff800`02fa5bb3 498122ff0f0000 and qword ptr [r10],0FFFh

CPU_COUNT: 4

CPU_MHZ: bfa

CPU_VENDOR: GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 25

CPU_STEPPING: 5

CPU_MICROCODE: 6,25,5,0 (F,M,S,R) SIG: 2'00000000 (cache) 2'00000000 (init)

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT

BUGCHECK_STR: 0xA

PROCESS_NAME: ekrn.exe

ANALYSIS_SESSION_HOST: DESKTOP-6P18NJQ

ANALYSIS_SESSION_TIME: 05-01-2019 17:39:30.0801

ANALYSIS_VERSION: 10.0.17763.132 amd64fre

TRAP_FRAME: fffff880070c38a0 -- (.trap 0xfffff880070c38a0) NOTE: The trap frame does not contain all registers. Some register values may be zeroed or incorrect. rax=000000001c92c009 rbx=0000000000000000 rcx=000000001c92d000 rdx=000000001c92d000 rsi=0000000000000000 rdi=0000000000000000 rip=fffff80002fa5bb3 rsp=fffff880070c3a30 rbp=fffff880070c3ca0 r8=fffffa8000e22600 r9=0000000000000000 r10=fffff88008e41000 r11=fffff70001146a70 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl nz na pe nc nt!MiGetWorkingSetInfo+0x263: fffff80002fa5bb3 498122ff0f0000 and qword ptr [r10],0FFFh ds:fffff88008e41000=???????????????? Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80002ea3d69 to fffff80002e959a0

STACK_TEXT:
fffff880070c3758 fffff80002ea3d69 : 000000000000000a fffff88008e41000 0000000000000000 0000000000000001 : nt!KeBugCheckEx fffff880070c3760 fffff80002ea1b88 : 0000000000000001 fffff88008e41000 0000000000000000 fffffa800ac0eb50 : nt!KiBugCheckDispatch+0x69 fffff880070c38a0 fffff80002fa5bb3 : fffffa800ac0eb50 00700001bd704867 fffffa800ac0eb50 0000000000000002 : nt!KiPageFault+0x448 fffff880070c3a30 fffff8000319b5e9 : 000000007ffeffff fffff80002f71f01 fffffa800ac0eb50 fffffa8000000001 : nt!MiGetWorkingSetInfo+0x263 fffff880070c3ac0 fffff80002ea39d3 : 000000000000029c fffffa800ac0eb50 000000000c41df78 000000000c41e288 : nt! ?? ::NNGAKEGL::string'+0x3c3e9 fffff880070c3bb0 0000000077a59aba : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiSystemServiceCopyEnd+0x13 000000000c41e268 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 00000000`00000000 : 0x77a59aba

THREAD_SHA1_HASH_MOD_FUNC: e29c6c26ff7f9c53bfbee68da1ffec088ec4d8fe

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: d303826312cdeb7a3cb5cc632e3a22096955af7b

THREAD_SHA1_HASH_MOD: ee8fcf1fb60cb6e3e2f60ddbed2ec02b5748a693

FOLLOWUP_IP: nt!MiGetWorkingSetInfo+263 fffff800`02fa5bb3 498122ff0f0000 and qword ptr [r10],0FFFh

FAULT_INSTR_CODE: ff228149

SYMBOL_STACK_INDEX: 3

SYMBOL_NAME: nt!MiGetWorkingSetInfo+263

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

DEBUG_FLR_IMAGE_TIMESTAMP: 5b94669f

IMAGE_VERSION: 6.1.7601.24260

STACK_COMMAND: .thread ; .cxr ; kb

IMAGE_NAME: memory_corruption

FAILURE_BUCKET_ID: X64_0xA_nt!MiGetWorkingSetInfo+263

BUCKET_ID: X64_0xA_nt!MiGetWorkingSetInfo+263

PRIMARY_PROBLEM_CLASS: X64_0xA_nt!MiGetWorkingSetInfo+263

TARGET_TIME: 2019-05-01T10:27:36.000Z

OSBUILD: 7601

OSSERVICEPACK: 1000

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 272

PRODUCT_TYPE: 1

OSPLATFORM_TYPE: x64

OSNAME: Windows 7

OSEDITION: Windows 7 WinNt (Service Pack 1) TerminalServer SingleUserTS

OS_LOCALE:

USER_LCID: 0

OSBUILD_TIMESTAMP: 2018-09-09 07:17:35

BUILDDATESTAMP_STR: 180908-0600

BUILDLAB_STR: win7sp1_ldr

BUILDOSVER_STR: 6.1.7601.24260.amd64fre.win7sp1_ldr.180908-0600

ANALYSIS_SESSION_ELAPSED_TIME: 3db

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:x64_0xa_nt!migetworkingsetinfo+263

FAILURE_ID_HASH: {94d36733-2a1b-7fa2-1b5d-36aebd04eeb5}

Followup: MachineOwner

YangKi1902 commented 5 years ago

im tried to terminate avp.exe process and no more BSOD, i think Kaspersky Antivirus make force exit of the hypervisor.

tanduRE commented 5 years ago

Kaspersky has a hypervisor for LSTAR hooking turn off screenshot protection / banking protection / secure browser or shutdown the hypervisor with vmcall

tandasat commented 5 years ago

Thank you for that info @tanduRE.

I have not been able to take time due to traveling. @InvisK -- does his solution workable for you, or do you expect me anything else?

YangKi1902 commented 5 years ago

hello, thanks for the response, i think it will working as im terminated KIS avp.exe host process, will try to shutdown the hypervisor of kaspersky programatically, thank you.