williballenthin / python-evtx

Pure Python parser for Windows Event Log files (.evtx)
Apache License 2.0
732 stars 166 forks source link

KeyError: 138 #41

Open patatetom opened 7 years ago

patatetom commented 7 years ago

hi Willy,

I've this issue with evtx_dump.py on Microsoft-Windows-Ntfs%4Operational.evtx file (from Windows 10) :

Traceback (most recent call last):
  File "/usr/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/usr/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/usr/lib/python3.6/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 191, in render_root_node
    return render_root_node_with_subs(root_node, subs)
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 176, in render_root_node_with_subs
    rec(c, acc)
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 166, in rec
    sub = render_root_node(sub.root())
  File "/usr/lib/python3.6/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/usr/lib/python3.6/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/Evtx/Nodes.py", line 1011, in substitutions
    self._chunk, self, type_, length=size)
  File "/usr/lib/python3.6/site-packages/Evtx/Nodes.py", line 468, in get_variant_value
    TypeClass = types[type_]
KeyError: 138

the evtx file Microsoft-Windows-Ntfs%4Operational.evtx seems to be ok because it is correctly exported from Windows : I send you the exported and the original evtx file...

regards, lacsaP.

williballenthin commented 7 years ago

i can reproduce this issue using the provided .evtx file, though unfortunately, my local system doesn't exhibit this unexpected node type. i'll continue to search for public evtx files that demonstrate the issue.

williballenthin commented 7 years ago

note:

hex(138) == 0x8A

this is possibly array (0x80) of QWORD (0x0A)

0x80 | 0x0A == 0x8A
williballenthin commented 7 years ago

this has been tricky to track down. still working on it.

patatetom commented 7 years ago

In its opening momentum, Microsoft has not documented its file format ?

limonidisicilia commented 6 years ago

Hi Willy I have exactly the same issue (can send Microsoft-Windows-Ntfs%4Operational.evtx).

Traceback (most recent call last):
  File "./evtx_dump.py", line 42, in <module>
    main()
  File "./evtx_dump.py", line 37, in main
    print(record.xml())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 191, in render_root_node
    return render_root_node_with_subs(root_node, subs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 176, in render_root_node_with_subs
    rec(c, acc)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 166, in rec
    sub = render_root_node(sub.root())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Nodes.py", line 1011, in substitutions
    self._chunk, self, type_, length=size)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Evtx/Nodes.py", line 468, in get_variant_value
    TypeClass = types[type_]
KeyError: 138

Thank you.

williballenthin commented 6 years ago

thanks also to @limonidisicilia for providing a sample .evtx file.

i'm able to reproduce this issue, but haven't yet figured out how the data is supposed to be parsed.

the immediate "solution" is to register an exception handler, log the raw data, and skip the record. long term, I might be able to figure out the binary format (and/or will happily accept patches!).

limonidisicilia commented 6 years ago

python-evtx seems to struggle with many WindowsX evtx files, to name a few: -Microsoft-Windows-Kernel-PnP%4Configuration.evtx -Microsoft-Windows-Storage-ClassPnP%4Operational.evtx -Microsoft-Windows-ApplicationResourceManagementSystem%4Operational.evtx' -Microsoft-Windows-AppModel-Runtime%4Admin.evtx

as a workaround I'm using Powershell: Get-WinEvent -path .\$item | foreach {$_.ToXml()} > "$item.xml"