search

williballenthin / EVTXtract

EVTXtract recovers and reconstructs fragments of EVTX log files from raw binary data, including unallocated space and memory images.
Apache License 2.0
189 stars 22 forks source link

New issue reading supposedly valid evtx file #5

Closed johnmccash closed 7 years ago

johnmccash commented 10 years ago

Hey Willi, I was just trying out the new version, and while running the initial 'extract_valid_evtx_records_and_templates' pass on the valid extracted evtx files from a host, I hit the following error. Thanks John

2014-10-07 13:29:37,723 WARNING root Flushing the existing template file due to exception.
2014-10-07 13:29:37,723 WARNING root Flushing the existing state file due to exception.
Traceback (most recent call last):
  File "extract_valid_evtx_records_and_templates.py", line 182, in main
    extract_valid_evtx_records_and_templates(state, templates, buf, progress_class=args.progress_class)
  File "extract_valid_evtx_records_and_templates.py", line 166, in extract_valid_evtx_records_and_templates
    extract_chunk(buf, chunk_offset, state, templates)
  File "extract_valid_evtx_records_and_templates.py", line 134, in extract_chunk
    for record in chunk.records():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 414, in records
    record = self.first_record()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 411, in first_record
    return Record(self._buf, self._offset + 0x200, self)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 437, in __init__
    raise InvalidRecordException()
InvalidRecordException: Parse Exception(Invalid record structure)
Traceback (most recent call last):
  File "extract_valid_evtx_records_and_templates.py", line 189, in <module>
    main()
  File "extract_valid_evtx_records_and_templates.py", line 182, in main
    extract_valid_evtx_records_and_templates(state, templates, buf, progress_class=args.progress_class)
  File "extract_valid_evtx_records_and_templates.py", line 166, in extract_valid_evtx_records_and_templates
    extract_chunk(buf, chunk_offset, state, templates)
  File "extract_valid_evtx_records_and_templates.py", line 134, in extract_chunk
    for record in chunk.records():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 414, in records
    record = self.first_record()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 411, in first_record
    return Record(self._buf, self._offset + 0x200, self)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 437, in __init__
    raise InvalidRecordException()
Evtx.Evtx.InvalidRecordException: Parse Exception(Invalid record structure)
johnmccash commented 10 years ago

When I tried to go on to processing the original image file, that barfed similarly. I tried using --vverbose, and the last few lines are as follows:

2014-10-07 16:55:42,440 DEBUG extract_records Considering chunk at offset 13220790272
2014-10-07 16:55:42,441 DEBUG extract_records Considering chunk at offset 13220859904
2014-10-07 16:55:42,442 DEBUG template_database Using existing template file: <case name redacted>_project.db.json
2014-10-07 16:55:42,466 WARNING root Flushing the existing template file due to exception.
2014-10-07 16:55:43,922 WARNING root Flushing the existing state file due to exception.
Traceback (most recent call last):
  File "extract_valid_evtx_records_and_templates.py", line 182, in main
    extract_valid_evtx_records_and_templates(state, templates, buf, progress_class=args.progress_class)
  File "extract_valid_evtx_records_and_templates.py", line 166, in extract_valid_evtx_records_and_templates
    extract_chunk(buf, chunk_offset, state, templates)
  File "extract_valid_evtx_records_and_templates.py", line 134, in extract_chunk
    for record in chunk.records():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 414, in records
    record = self.first_record()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 411, in first_record
    return Record(self._buf, self._offset + 0x200, self)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 437, in __init__
    raise InvalidRecordException()
InvalidRecordException: Parse Exception(Invalid record structure)
Traceback (most recent call last):
  File "extract_valid_evtx_records_and_templates.py", line 189, in <module>
    main()
  File "extract_valid_evtx_records_and_templates.py", line 182, in main
    extract_valid_evtx_records_and_templates(state, templates, buf, progress_class=args.progress_class)
  File "extract_valid_evtx_records_and_templates.py", line 166, in extract_valid_evtx_records_and_templates
    extract_chunk(buf, chunk_offset, state, templates)
  File "extract_valid_evtx_records_and_templates.py", line 134, in extract_chunk
    for record in chunk.records():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 414, in records
    record = self.first_record()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 411, in first_record
    return Record(self._buf, self._offset + 0x200, self)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Evtx.py", line 437, in __init__
    raise InvalidRecordException()
Evtx.Evtx.InvalidRecordException: Parse Exception(Invalid record structure)

This looks almost exactly like the other problem that you supposedly just put in a workaround for, except that the unknown exception and opening/ending tag mismatch messages are missing, right before the "Using existing template file" message. Are you sure you suppressed that error completely? Thanks John

williballenthin commented 7 years ago

I've recently rewritten much of EVTXtract to avoid bugs like these. Please retry on the latest version. This bug should be fixed.