search

tklengyel / drakvuf

DRAKVUF Black-box Binary Analysis
https://drakvuf.com
Other
1.06k stars 255 forks source link

Invalid guest state after attach to Windows 10 VM #465

Closed skvl closed 6 years ago

skvl commented 6 years ago

Hello.

I experience a stable bug on my system: a DomU crashes after drakvuf attaches to it.

My host system

My drakvuf info

My DomU info

vcpus = 1 maxvcpus = 1

memory = 2048 maxmem = 2048

on_poweroff = "destroy" on_reboot = "restart" on_watchdog = "preserve" on_crash = "preserve" on_soft_reset = "soft-reset"

disk = [ "format=qcow2, vdev=hda, target=/mnt/images/win10-1803-x64/image.qcow2" ]

vif = [ "bridge=virbr0, type=ioemu, model=e1000, mac=4c:72:b9:31:88:b1" ]

hdtype = "ahci"

shadow_memory = 16

altp2m = "external"

viridian = [ "all" ]

videoram = 128 vga = "stdvga"

vnc = 1 vnclisten="0.0.0.0:1"

soundhw = "hda"

usb = 1 usbdevice = [ "tablet" ]

serial = [ 'unix:/tmp/xen-debuggee.pipe' ]


# *drakvuf* log

sudo drakvuf -r /mnt/images/win10-1803-x64/rekall-kernel.json -d 10 -v 1536731122.795691 DRAKVUF v0.6-a4f3575 1536731122.795783 Starting DRAKVUF initialization 1536731122.882763 drakvuf_event_fd_add fd=14 1536731122.882791 size of list=1 1536731122.882804 regenerating event_fds and fd_info_lookup... 1536731122.882819 new event_fd i=0 for fd=14 1536731122.882831 new fd_info_lookup i=0 for fd=14 1536731122.882842 drakvuf_init: adding event_fd done 1536731122.883161 Init VMI on domID 10 -> TRAP 1536731122.884046 init_vmi: initializing vmi done 1536731123.467308 Max GPFN: 0xff001 1536731123.467347 Max mem set? 0 1536731123.467394 Physmap populated? 0 1536731123.467547 Altp2m enabled? 0 1536731123.467565 Altp2m view X created? 0 with ID 1 1536731123.467583 Altp2m view R created? 0 with ID 2 1536731123.467615 Switched Altp2m view to X? 0 1536731123.518041 Windows kernel base address is 0xfffff803b7000000 1536731123.518069 Rekall profile: symbol 'KiInitialPCR' not found 1536731123.518091 Failed to find offset for KiInitialPCR:(null) 1536731123.518118 Rekall profile: '_KPCR' has no 'PrcbData' member 1536731123.518130 Failed to find offset for _KPCR:PrcbData 1536731123.518168 Failed to find offsets for array of structure names and subsymbols. 1536731123.518185 libdrakvuf initialized 1536731123.518203 DRAKVUF initializated 1536731123.518229 Starting plugins 1536731123.518245 Starting plugin syscalls 1536731123.518262 Starting plugin syscalls finished 1536731123.518280 Starting plugin poolmon 1536731123.518298 Starting plugin poolmon finished 1536731123.518316 Starting plugin filetracer 1536731123.518333 Starting plugin filetracer finished 1536731123.518351 Starting plugin filedelete 1536731123.518368 Starting plugin filedelete finished 1536731123.518387 Starting plugin objmon 1536731123.518404 Starting plugin objmon finished 1536731123.518422 Starting plugin exmon 1536731123.518439 Starting plugin exmon finished 1536731123.518451 Starting plugin ssdtmon 1536731123.518463 Starting plugin ssdtmon finished 1536731123.518474 Starting plugin debugmon 1536731123.518486 Starting plugin debugmon finished 1536731123.518499 Starting plugin delaymon 1536731123.518509 Starting plugin delaymon finished 1536731123.518521 Starting plugin cpuidmon 1536731123.518533 Starting plugin cpuidmon finished json_object_from_file: error opening file (null): Bad address 1536731123.518667 Starting plugin socketmon 1536731123.518675 Starting plugin socketmon finished 1536731123.518692 Starting plugin regmon 1536731123.518709 Starting plugin regmon finished 1536731123.518727 Starting plugin procmon 1536731123.518760 Starting plugin procmon finished 1536731123.518777 Starting plugin bsodmon 1536731123.518795 Starting plugin bsodmon finished 1536731123.518812 Beginning DRAKVUF loop 1536731123.518830 Started DRAKVUF loop


# *Xen* info

## *xl list* after crash

TRAP 10 2048 1 ---sc- 87.7


## *xl dmesg*

(XEN) HVM10 save: CPU (XEN) HVM10 save: PIC (XEN) HVM10 save: IOAPIC (XEN) HVM10 save: LAPIC (XEN) HVM10 save: LAPIC_REGS (XEN) HVM10 save: PCI_IRQ (XEN) HVM10 save: ISA_IRQ (XEN) HVM10 save: PCI_LINK (XEN) HVM10 save: PIT (XEN) HVM10 save: RTC (XEN) HVM10 save: HPET (XEN) HVM10 save: PMTIMER (XEN) HVM10 save: MTRR (XEN) HVM10 save: VIRIDIAN_DOMAIN (XEN) HVM10 save: CPU_XSAVE (XEN) HVM10 save: VIRIDIAN_VCPU (XEN) HVM10 save: VMCE_VCPU (XEN) HVM10 save: TSC_ADJUST (XEN) HVM10 save: CPU_MSR (XEN) HVM10 restore: CPU 0 (d10) HVM Loader (d10) Detected Xen v4.9.2 (d10) Xenbus rings @0xfeffc000, event channel 1 (d10) System requested SeaBIOS (d10) CPU speed is 3392 MHz (d10) Relocating guest memory for lowmem MMIO space disabled (d10) PCI-ISA link 0 routed to IRQ5 (d10) PCI-ISA link 1 routed to IRQ10 (d10) PCI-ISA link 2 routed to IRQ11 (d10) PCI-ISA link 3 routed to IRQ5 (d10) pci dev 01:2 INTD->IRQ5 (d10) pci dev 01:3 INTA->IRQ10 (d10) pci dev 02:0 INTA->IRQ11 (d10) pci dev 03:0 INTA->IRQ5 (d10) pci dev 05:0 INTA->IRQ10 (d10) pci dev 06:0 INTA->IRQ11 (d10) No RAM in high memory; setting high_mem resource base to 100000000 (d10) pci dev 04:0 bar 10 size 008000000: 0f0000008 (d10) pci dev 02:0 bar 14 size 001000000: 0f8000008 (d10) pci dev 05:0 bar 30 size 000080000: 0f9000000 (d10) pci dev 05:0 bar 10 size 000020000: 0f9080000 (d10) pci dev 04:0 bar 30 size 000010000: 0f90a0000 (d10) pci dev 03:0 bar 10 size 000004000: 0f90b0000 (d10) pci dev 04:0 bar 18 size 000001000: 0f90b4000 (d10) pci dev 06:0 bar 24 size 000001000: 0f90b5000 (d10) pci dev 02:0 bar 10 size 000000100: 00000c001 (d10) pci dev 05:0 bar 14 size 000000040: 00000c101 (d10) pci dev 01:2 bar 20 size 000000020: 00000c141 (d10) pci dev 06:0 bar 20 size 000000020: 00000c161 (d10) pci dev 01:1 bar 20 size 000000010: 00000c181 (d10) Multiprocessor initialisation: (d10) - CPU0 ... 39-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done. (d10) Writing SMBIOS tables ... (d10) Loading SeaBIOS ... (d10) Creating MP tables ... (d10) Loading ACPI ... (d10) CONV disabled (d10) vm86 TSS at fc00a600 (d10) BIOS map: (d10) 10000-100e3: Scratch space (d10) c0000-fffff: Main BIOS (d10) E820 table: (d10) [00]: 00000000:00000000 - 00000000:000a0000: RAM (d10) HOLE: 00000000:000a0000 - 00000000:000c0000 (d10) [01]: 00000000:000c0000 - 00000000:00100000: RESERVED (d10) [02]: 00000000:00100000 - 00000000:78000000: RAM (d10) HOLE: 00000000:78000000 - 00000000:fc000000 (d10) [03]: 00000000:fc000000 - 00000001:00000000: RESERVED (d10) Invoking SeaBIOS ... (d10) SeaBIOS (version 1.10.2-1ubuntu1) (d10) BUILD: gcc: (Ubuntu 6.3.0-16ubuntu6) 6.3.0 20170506 binutils: (GNU Binutils for (d10) Ubuntu) 2.28 (d10) (d10) Found Xen hypervisor signature at 40000100 (d10) Running on QEMU (i440fx) (d10) xen: copy e820... (d10) Relocating init from 0x000da800 to 0x77face20 (size 78144) (d10) Found 10 PCI devices (max PCI bus is 00) (d10) Allocated Xen hypercall page at 77fff000 (d10) Detected Xen v4.9.2 (d10) xen: copy BIOS tables... (d10) Copying SMBIOS entry point from 0x00010020 to 0x000f6be0 (d10) Copying MPTABLE from 0xfc001130/fc001140 to 0x000f6b00 (d10) Copying PIR from 0x00010040 to 0x000f6a80 (d10) Copying ACPI RSDP from 0x000100c0 to 0x000f6a50 (d10) Using pmtimer, ioport 0xb008 (d10) Scan for VGA option rom (d10) Running option rom at c000:0003 (d10) Turning on vga text mode console (d10) SeaBIOS (version 1.10.2-1ubuntu1) (d10) Machine UUID 0fd25d55-539b-4aab-a0d3-fdfc86ad5482 (d10) UHCI init on dev 00:01.2 (io=c140) (d10) ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9) (d10) ATA controller 2 at 170/374/0 (irq 15 dev 9) (d10) AHCI controller at 00:06.0, iobase 0xf90b5000, irq 11 (d10) Searching bootorder for: /pci@i0cf8/@6/drive@0/disk@0 (d10) AHCI/0: Set transfer mode to UDMA-5 (d10) AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (931 GiBytes)" (d10) PS2 keyboard initialized (d10) Found 0 lpt ports (d10) Found 1 serial ports (d10) All threads complete. (d10) Scan for option roms (d10) Running option rom at c980:0003 (d10) pmm call arg1=1 (d10) pmm call arg1=0 (d10) pmm call arg1=1 (d10) pmm call arg1=0 (d10) Searching bootorder for: /pci@i0cf8/@5 (d10) (d10) Press ESC for boot menu. (d10) (d10) Searching bootorder for: HALT (d10) drive 0x000f69a0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=195352516 (d10) 8 (d10) Space available for UMB: ca800-ec800, f6400-f69a0 (d10) Returned 253952 bytes of ZoneHigh (d10) e820 map has 6 items: (d10) 0: 0000000000000000 - 000000000009fc00 = 1 RAM (d10) 1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED (d10) 2: 00000000000f0000 - 0000000000100000 = 2 RESERVED (d10) 3: 0000000000100000 - 0000000077ffe000 = 1 RAM (d10) 4: 0000000077ffe000 - 0000000078000000 = 2 RESERVED (d10) 5: 00000000fc000000 - 0000000100000000 = 2 RESERVED (d10) enter handle_19: (d10) NULL (d10) Booting from Hard Disk... (d10) Booting from 0000:7c00 (XEN) d10: VIRIDIAN GUEST_OS_ID: vendor: 1 os: 4 major: a minor: 0 sp: 0 build: 271b (XEN) d10: VIRIDIAN HYPERCALL: enabled: 1 pfn: 20f (XEN) d10: VIRIDIAN MSR_TIME_REF_COUNT: accessed (XEN) d10: VIRIDIAN REFERENCE_TSC: enabled: 1 pfn: c (XEN) d10v0: VIRIDIAN VP_ASSIST_PAGE: enabled: 1 pfn: d (XEN) d10v0 vmentry failure (reason 0x80000021): Invalid guest state (0) (XEN) *** VMCS Area ** (XEN) * Guest State (XEN) CR0: actual=0x0000000080050031, shadow=0x0000000080050031, gh_mask=ffffffffffffffff (XEN) CR4: actual=0x0000000000172678, shadow=0x0000000000170678, gh_mask=ffffffffffffffff (XEN) CR3 = 0x800000001b6b0002 (XEN) PDPTE0 = 0x0000000000000000 PDPTE1 = 0x000022e600022000 (XEN) PDPTE2 = 0x0000000000150000 PDPTE3 = 0x000026e000152000 (XEN) RSP = 0x800000001b6b0002 (0x800000001b6b0002) RIP = 0xfffff803b72de124 (0xfffff803b72de124) (XEN) RFLAGS=0x00000046 (0x00000046) DR7 = 0x0000000000000400 (XEN) Sysenter RSP=0000000000000000 CS:RIP=0000:0000000000000000 (XEN) sel attr limit base (XEN) CS: 0010 0a09b ffffffff 0000000000000000 (XEN) DS: 002b 0c0f3 ffffffff 0000000000000000 (XEN) SS: 0018 0c093 ffffffff 0000000000000000 (XEN) ES: 002b 0c0f3 ffffffff 0000000000000000 (XEN) FS: 0053 040f3 00003c00 0000000000000000 (XEN) GS: 002b 0c0f3 ffffffff fffff803b63c5000 (XEN) GDTR: 00000057 fffff803ba453fb0 (XEN) LDTR: 0000 1c000 ffffffff 0000000000000000 (XEN) IDTR: 00000fff fffff803ba451000 (XEN) TR: 0040 0008b 00000067 fffff803ba452000 (XEN) EFER = 0x0000000000000000 PAT = 0x0007010600070106 (XEN) PreemptionTimer = 0x00000000 SM Base = 0x00000000 (XEN) DebugCtl = 0x0000000000000000 DebugExceptions = 0x0000000000000000 (XEN) Interruptibility = 00000000 ActivityState = 00000000 (XEN) Host State (XEN) RIP = 0xffff82d080309dc0 (vmx_asm_vmexit_handler) RSP = 0xffff830431307f70 (XEN) CS=e008 SS=0000 DS=0000 ES=0000 FS=0000 GS=0000 TR=e040 (XEN) FSBase=0000000000000000 GSBase=0000000000000000 TRBase=ffff83043130ac80 (XEN) GDTBase=ffff8304312fe000 IDTBase=ffff830431308000 (XEN) CR0=0000000080050033 CR3=000000039492d000 CR4=00000000001526e0 (XEN) Sysenter RSP=ffff830431307fa0 CS:RIP=e008:ffff82d08034b900 (XEN) EFER = 0x0000000000000000 PAT = 0x0000050100070406 (XEN) Control State * (XEN) PinBased=0000003f CPUBased=b6a0e5fa SecondaryExec=000054eb (XEN) EntryControls=000053ff ExitControls=000fefff (XEN) ExceptionBitmap=0006000a PFECmask=00000000 PFECmatch=00000000 (XEN) VMEntry: intr_info=0000002f errcode=00000004 ilen=00000000 (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 (XEN) reason=80000021 qualification=0000000000000000 (XEN) IDTVectoring: info=00000000 errcode=00000000 (XEN) TSC Offset = 0xfffff418a22ef5c2 TSC Multiplier = 0x0000000000000000 (XEN) TPR Threshold = 0x00 PostedIntrVec = 0x00 (XEN) EPT pointer = 0x000000039495301e EPTP index = 0x0000 (XEN) PLE Gap=00000080 Window=00001000 (XEN) Virtual processor ID = 0x7678 VMfunc controls = 0000000000000000 (XEN) **** (XEN) domain_crash called from vmx.c:3411 (XEN) Domain 10 (vcpu#0) crashed on cpu#6: (XEN) ----[ Xen-4.9.2 x86_64 debug=n Not tainted ]---- (XEN) CPU: 6 (XEN) RIP: 0010:[] (XEN) RFLAGS: 0000000000000046 CONTEXT: hvm guest (d10v0) (XEN) rax: 0000000000000087 rbx: 0000019993e1c690 rcx: 00007ffe423fb064 (XEN) rdx: 0000000000000000 rsi: 0000000000000158 rdi: 0000019993eaebf0 (XEN) rbp: 0000002a365ff9b0 rsp: 800000001b6b0002 r8: 0000002a365ff908 (XEN) r9: 0000000000000010 r10: 0000000000000158 r11: 0000000000000246 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 00007ffe3fb73c14 (XEN) r15: 0000000000000000 cr0: 0000000080050031 cr4: 0000000000170678 (XEN) cr3: 800000001b6b0002 cr2: 00007ffe1aa2a5c0 (XEN) fsb: 0000000000000000 gsb: fffff803b63c5000 gss: 000000f752ddf000 (XEN) ds: 002b es: 002b fs: 0053 gs: 002b ss: 0018 cs: 0010


## /var/log/xen/xl-TRAP.log

Waiting for domain TRAP (domid 10) to die [pid 14958] Domain 10 has shut down, reason code 3 0x3 Action for shutdown reason code 3 is preserve Done. Exiting now

skvl commented 6 years ago

@tklengyel let me know if I could gather more info.

aoshiken commented 6 years ago

Any chance of testing with the latest Xen v4.11?

skvl commented 6 years ago

I will try as soon as possible.

tklengyel commented 6 years ago

Yes, DRAKVUF needs Xen 4.11 as per the instructions. This bug has already been fixed there.