search

osquery / osquery

SQL powered operating system instrumentation, monitoring, and analytics.
https://osquery.io
Other
21.84k stars 2.45k forks source link

exception_code and module values are missing when querying windows_crashes table #6033

Open ian-mv opened 4 years ago

ian-mv commented 4 years ago

Bug report

What operating system and version are you using?

Windows 10 version 1809 build 17763.805

What version of osquery are you using?

4.02

What steps did you take to reproduce the issue?

  1. Triggered a system crash (by abruptly shutting down the host system running a VM as SUT)
  2. Log back into the VM
  3. Observe that exception_code and module values are empty in the output when windows_crashes table is queried
  4. Locate the crash dump and parse it using windbg
  5. Observe that exception_code and module information is present in the parsed crash dump

What did you expect to see?

exception_code and module values populated when windows_crashes table is queried

What did you see instead?

exception_code and module values empty when windows_crashes table is queried

Additional information

Query: 

select
  'System Crash' AS event_name,
  'complete' AS event_status,
  (select logon_domain || '/' || user
     from logon_sessions
     where user in (
       select user from logged_in_users where type='active' LIMIT(1)
     )
     LIMIT(1)
  ) as user,
  path AS process,
  version As version,
  module AS module,
  exception_code as error,
  stack_trace AS stack_trace,
  cast((strftime('%s',split(datetime, 'UTC', 0)) * 1000000000) as int) as trigger_time
from windows_crashes
where crash_path in (
  select path
  from file
  where
    path like (
      select (data || 'windows\\minidump\\%.dmp') as path
      from registry
      where path='$path1'
    )
    or path like (
      select (data || 'windows\\memory.dmp%') as path
      from registry
      where path='$path2'
    )
);

$path1 = "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\BootDir"
$path2 = "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\BootDir"

Parsed crash dump:

Microsoft (R) Windows Debugger Version 10.0.17763.132 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Windows\Minidump\110719-10937-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

************* Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*C:\\ProgramData\\dbg\\sym*https://msdl.microsoft.com/download/symbols
Symbol search path is: srv*C:\\ProgramData\\dbg\\sym*https://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 10 Kernel Version 17763 MP (2 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 17763.1.amd64fre.rs5_release.180914-1434
Machine Name:
Kernel base = 0xfffff800`460b3000 PsLoadedModuleList = 0xfffff800`464ce690
Debug session time: Wed Nov  6 21:36:51.814 2019 (UTC - 6:00)
System Uptime: 0 days 0:53:42.565
Loading Kernel Symbols
...............................................................
................................................................
............................................................
Loading User Symbols
Loading unloaded module list
.......
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 5C, {115, fffff7e5800154b0, b5cb, ffffffffc0000001}

*** WARNING: Unable to verify timestamp for win32k.sys
*** ERROR: Module load completed but symbols could not be loaded for win32k.sys
Probably caused by : ntkrnlmp.exe ( nt!KiCallInterruptServiceRoutine+a5 )

Followup:     MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

HAL_INITIALIZATION_FAILED (5c)
Arguments:
Arg1: 0000000000000115
Arg2: fffff7e5800154b0
Arg3: 000000000000b5cb
Arg4: ffffffffc0000001

Debugging Details:
------------------

KEY_VALUES_STRING: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 400

BUILD_VERSION_STRING:  17763.1.amd64fre.rs5_release.180914-1434

SYSTEM_MANUFACTURER:  VMware, Inc.

VIRTUAL_MACHINE:  VMware

SYSTEM_PRODUCT_NAME:  VMware7,1

SYSTEM_VERSION:  None

BIOS_VENDOR:  VMware, Inc.

BIOS_VERSION:  VMW71.00V.12343141.B64.1902160724

BIOS_DATE:  02/16/2019

BASEBOARD_MANUFACTURER:  Intel Corporation

BASEBOARD_PRODUCT:  440BX Desktop Reference Platform

BASEBOARD_VERSION:  None

DUMP_TYPE:  2

BUGCHECK_P1: 115

BUGCHECK_P2: fffff7e5800154b0

BUGCHECK_P3: b5cb

BUGCHECK_P4: ffffffffc0000001

ADDITIONAL_DEBUG_TEXT:  Halfailures structure in bugcheck.cpp needs updated.  Please information OCA (alias:werka).

CPU_COUNT: 2

CPU_MHZ: a20

CPU_VENDOR:  GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 9e

CPU_STEPPING: a

CPU_MICROCODE: 6,9e,a,0 (F,M,S,R)  SIG: B4'00000000 (cache) B4'00000000 (init)

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXPNP: 1 (!blackboxpnp)

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  0x5C

PROCESS_NAME:  System

CURRENT_IRQL:  f

ANALYSIS_SESSION_HOST:  DESKTOP-2CJHG4K

ANALYSIS_SESSION_TIME:  11-07-2019 08:56:38.0486

ANALYSIS_VERSION: 10.0.17763.132 amd64fre

EXCEPTION_RECORD:  fffff800462350bb -- (.exr 0xfffff800462350bb)
ExceptionAddress: ccccccc328c48348
   ExceptionCode: 50247c83
  ExceptionFlags: 72850f00
NumberParameters: -858993460
   Parameter[0]: 8948000000cd840f
   Parameter[1]: 182474894808245c
   Parameter[2]: 48ec8b4856415755
   Parameter[3]: 3ba3058b4870ec83
   Parameter[4]: 458948c433480028
   Parameter[5]: 48f18b41f18b4cf8
   Parameter[6]: da8b48f88b49ca8b
   Parameter[7]: d08b48fff11e54e8
   Parameter[8]: 4c18488b4ccb8b48
   Parameter[9]: ffeab301e8d84d89
   Parameter[10]: 7d21487f75ff8548
   Parameter[11]: 8900000098838be0
   Parameter[12]: 4c404538c033ec45
   Parameter[13]: 508de87589d07589
   Parameter[14]: 00487d80c2450f01

LAST_CONTROL_TRANSFER:  from fffff800460609f7 to fffff80046267050

STACK_TEXT:  
fffff800`48874eb8 fffff800`460609f7 : 00000000`0000005c 00000000`00000115 fffff7e5`800154b0 00000000`0000b5cb : nt!KeBugCheckEx
fffff800`48874ec0 fffff800`46060372 : 00000018`b6e88de9 00000000`0000b5ca fffff7e5`80016380 fffff800`48865590 : hal!HalpVpptUpdatePhysicalTimer+0x143
fffff800`48874f00 fffff800`46016276 : 00000018`b6e88de9 fffff800`4607ff80 fffff800`48865610 00000000`00000000 : hal!HalpVpptAcknowledgeInterrupt+0x102
fffff800`48874f30 fffff800`461e63a5 : 00000007`80d0b005 fffff800`4607ff80 fffff800`46080030 ffff7241`7b7fd8ea : hal!HalpTimerClockInterrupt+0x36
fffff800`48874f60 fffff800`46268a5a : fffff800`48865610 fffff800`4607ff80 ffffb48c`ba2c8140 fffff800`4607ff80 : nt!KiCallInterruptServiceRoutine+0xa5
fffff800`48874fb0 fffff800`46268fa7 : 00000000`00000009 fffff800`48865610 fffff800`4607ff80 fffff800`46016945 : nt!KiInterruptSubDispatchNoLockNoEtw+0xfa
fffff800`48865590 fffff800`4602e9bf : fffff800`462350bb 00000000`00000000 ffffb48c`ba2c8140 fffff800`451c2180 : nt!KiInterruptDispatchNoLockNoEtw+0x37
fffff800`48865728 fffff800`462350bb : 00000000`00000000 ffffb48c`ba2c8140 fffff800`451c2180 ffffb48c`ba2c8050 : hal!HalProcessorIdle+0xf
fffff800`48865730 fffff800`46199b7b : 00000000`00000000 00000000`00000000 fffff800`48865802 00000000`00004386 : nt!PpmIdleDefaultExecute+0x1b
fffff800`48865760 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!PpmIdleExecuteTransition+0x6bb

THREAD_SHA1_HASH_MOD_FUNC:  fa701739e0c7175431937cec390b79345998fc7b

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  928eb477bf5bed495cdb9cf4f60ff16599177290

THREAD_SHA1_HASH_MOD:  9f36ced8edf7721fc2f3af352fe81bdb59b775c4

FOLLOWUP_IP: 
nt!KiCallInterruptServiceRoutine+a5
fffff800`461e63a5 0fb6e8          movzx   ebp,al

FAULT_INSTR_CODE:  45e8b60f

SYMBOL_STACK_INDEX:  4

SYMBOL_NAME:  nt!KiCallInterruptServiceRoutine+a5

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  320760b6

IMAGE_VERSION:  10.0.17763.802

STACK_COMMAND:  .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET:  a5

FAILURE_BUCKET_ID:  0x5C_nt!KiCallInterruptServiceRoutine

BUCKET_ID:  0x5C_nt!KiCallInterruptServiceRoutine

PRIMARY_PROBLEM_CLASS:  0x5C_nt!KiCallInterruptServiceRoutine

TARGET_TIME:  2019-11-07T03:36:51.000Z

OSBUILD:  17763

OSSERVICEPACK:  802

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  272

PRODUCT_TYPE:  1

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

OSEDITION:  Windows 10 WinNt TerminalServer SingleUserTS

OS_LOCALE:  

USER_LCID:  0

OSBUILD_TIMESTAMP:  1996-08-06 10:11:50

BUILDDATESTAMP_STR:  180914-1434

BUILDLAB_STR:  rs5_release

BUILDOSVER_STR:  10.0.17763.1.amd64fre.rs5_release.180914-1434

ANALYSIS_SESSION_ELAPSED_TIME:  2eef

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:0x5c_nt!kicallinterruptserviceroutine

FAILURE_ID_HASH:  {8e155cd0-c545-25d5-7ee1-936e57893c8c}

Followup:     MachineOwner
---------

0: kd> lmvm nt
Browse full module list
start             end                 module name
fffff800`460b3000 fffff800`46b23000   nt         (pdb symbols)          c:\\programdata\\dbg\\sym\ntkrnlmp.pdb\3B5C345ED2538B4C4A412409A0FDF4351\ntkrnlmp.pdb
    Loaded symbol image file: ntkrnlmp.exe
    Mapped memory image file: c:\\programdata\\dbg\\sym\ntoskrnl.exe\320760B6a70000\ntoskrnl.exe
    Image path: ntkrnlmp.exe
    Image name: ntkrnlmp.exe
    Browse all global symbols  functions  data
    Image was built with /Brepro flag.
    Timestamp:        320760B6 (This is a reproducible build file hash, not a timestamp)
    CheckSum:         0094767A
    ImageSize:        00A70000
    File version:     10.0.17763.802
    Product version:  10.0.17763.802
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        1.0 App
    File date:        00000000.00000000
    Translations:     0409.04b0
    Information from resource tables:
        CompanyName:      Microsoft Corporation
        ProductName:      Microsoft® Windows® Operating System
        InternalName:     ntkrnlmp.exe
        OriginalFilename: ntkrnlmp.exe
        ProductVersion:   10.0.17763.802
        FileVersion:      10.0.17763.802 (WinBuild.160101.0800)
        FileDescription:  NT Kernel & System
        LegalCopyright:   © Microsoft Corporation. All rights reserved.
theopolis commented 4 years ago

We should consider refactoring the windows_crashes table. I looked briefly and I see a lot of missing error checking and re-initialization of resources. I do not know enough about windows to do this myself.