search

log2timeline / plaso

Super timeline all the things
https://plaso.readthedocs.io
Apache License 2.0
1.7k stars 334 forks source link

Issue with parsing/reading compressed files #90

Closed kiddinn closed 9 years ago

kiddinn commented 9 years ago

Running against a test image:

2015-01-21 13:18:16,817 [WARNING] (Worker_7  ) PID:27148 <worker> Unable to parse file: type:   OS, location: /PATH/test_images/win7-64-nfury-10.3.58.6/win7-64-nfury-c-drive/win7-64-nfury-c-  drive.E01
type: EWF
type: TSK, inode: 10, location: /$UpCase
 with error: FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "build/bdist.linux-x86_64/egg/plaso/multi_processing/multi_process.py", line 604, in run
    self.run()
  File "build/bdist.linux-x86_64/egg/plaso/multi_processing/multi_process.py", line 604, in run
    self._extraction_worker.Run()
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 370, in Run
    self._extraction_worker.Run()
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 370, in Run
    self.ConsumeItems()
  File "build/bdist.linux-x86_64/egg/plaso/engine/queue.py", line 171, in ConsumeItems
    self._ConsumeItem(item)
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 106, in _ConsumeItem
    self.ParseFileEntry(file_entry)
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 333, in ParseFileEntry
    **is_compressed_stream = self._ProcessCompressedStreamFile(file_entry)**
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 225, in     _ProcessCompressedStreamFile
    file_entry.path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 259, in     GetCompressedStreamTypeIndicators
    self.ConsumeItems()
  File "build/bdist.linux-x86_64/egg/plaso/engine/queue.py", line 171, in ConsumeItems
    cls._compressed_stream_remainder_list, path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 193, in _GetTypeIndicators
    file_object = resolver.Resolver.OpenFileObject(path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/resolver.py", line 118, in OpenFileObject
    self._ConsumeItem(item)
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 106, in _ConsumeItem
self.ParseFileEntry(file_entry)
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 333, in ParseFileEntry
    is_compressed_stream = self._ProcessCompressedStreamFile(file_entry)
  File "build/bdist.linux-x86_64/egg/plaso/engine/worker.py", line 225, in     _ProcessCompressedStreamFile
    file_entry.path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 259, in     GetCompressedStreamTypeIndicators
    file_object = resolver_helper.OpenFileObject(path_spec, resolver_context)
  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/tsk_resolver_helper.py", line 47, in OpenFileObject
    file_object.open(path_spec=path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/file_io/tsk_file_io.py", line 88, in open
    cls._compressed_stream_remainder_list, path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 193, in _GetTypeIndicators
    file_object = resolver.Resolver.OpenFileObject(path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/resolver.py", line 118, in OpenFileObject
    path_spec, resolver_context=self._resolver_context)
  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/resolver.py", line 165, in OpenFileSystem
    file_system = resolver_helper.OpenFileSystem(path_spec, resolver_context)
  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/tsk_resolver_helper.py", line 68, in     OpenFileSystem
    file_object = resolver_helper.OpenFileObject(path_spec, resolver_context)
    resolver_context, file_object, path_spec.parent)
  File "build/bdist.linux-x86_64/egg/dfvfs/vfs/tsk_file_system.py", line 54, in __init__
    self._tsk_file_system = pytsk3.FS_Info(tsk_image_object, offset=offset)

The error that comes up is: RuntimeError: 'pyewf_handle_read_buffer: unable to read data. libewf_handle_read_buffer: chunk offset exceeds chunk data size.'

Some of the more relevant portions of the traceback:

_File "...plaso/engine/worker.py", line 333, in ParseFileEntry *_is_compressed_stream = self._ProcessCompressedStreamFile(file_entry)***

_File "...plaso/engine/worker.py", line 225, in *__ProcessCompressedStreamFile***

This gets repeated quite frequently and is easily reproducible on the SANS NFURY test image.

joachimmetz commented 9 years ago

Can you try single process mode.

joachimmetz commented 9 years ago

Was this fixed by the fixes for issue #91 as well?

joachimmetz commented 9 years ago

Closing issue, reopen if this still happens.