Analysis plugin nsrlsvr: KeyError in plaso/analysis/interface.py

[kinky-it]

Plaso version:

1.5.0 Release

Operating system Plaso is running on:

XUbuntu 64 bit

Installation method:

Installed from GiFT

Description of problem:

Tagging events using the --analysis nsrlsvr option throws an exception.

Debug output/tracebacks:

The following error is shown in the debug log:

[ERROR] (nsrlsvr   ) PID:XXXXX <analysis_process> <dfvfs.path.os_path_spec.OSPathSpec object at 0xXXXXXXXXXXXX>
Traceback (most recent call last):                                                                                                     
  File "/usr/lib/python2.7/dist-packages/plaso/multi_processing/analysis_process.py", line 138, in _Main                               
    self._analysis_mediator.ProduceAnalysisReport(self._analysis_plugin)                                                               
  File "/usr/lib/python2.7/dist-packages/plaso/analysis/mediator.py", line 95, in ProduceAnalysisReport                                
    analysis_report = plugin.CompileReport(self)                                                                                       
  File "/usr/lib/python2.7/dist-packages/plaso/analysis/interface.py", line 280, in CompileReport                                      
    hash_analysis)                                                                                                                     
  File "/usr/lib/python2.7/dist-packages/plaso/analysis/interface.py", line 187, in _HandleHashAnalysis                                
    event_uuids = self._event_uuids_by_pathspec.pop(pathspec)                                                                          
KeyError: <dfvfs.path.os_path_spec.OSPathSpec object at 0xXXXXXXXXXXXX>

Source data:

The exception is caused by popping from _event_uuids_by_pathspec. This is a defaultdict, so retrieving an unexisting entry from the dict using the key would be no problem. A pop() is, though.

Problem can be fixed by catching a KeyError on line 187 of plaso/analysis/interface.py and set event_uuids to an empty list.

  try:
    event_uuids = self._event_uuids_by_pathspec.pop(pathspec)
  except KeyError:
    event_uuids = []

[pettai]

just out curiosity, did you ever get this to work? and which xubuntu version did you use? I tested this too (running xenial with ppa:gift/dev), but it didn't give me a hit at all.

https://gist.github.com/pettai/72d9bfbc901e4ae74a84454ccf3e550b

[kinky-it]

Ubuntu 14.04.4

You may need to check out #1036 (and my extra comment there) too.

[pettai]

Saw that, so I only used the md5 hasher and also generated my own md5 hash list for nsrlsvr, just to be sure that it should contain known files (see my gist paste).

[joachimmetz]

I only used the md5 hasher and also generated my own md5 hash list for nsrlsvr

# test: md5 hash exists
~$ md5sum plaso-1.5.0/LICENSE | nsrllookup

Make sure you're talking to the right server by default nsrllookup will query "nsrllookup.com"

# test: md5 hash exists
~$ md5sum plaso-1.5.0/LICENSE | nsrllookup -s 127.0.0.1 -p 9120 -k

[joachimmetz]

Traceback (most recent call last):
  File "multi_processing/analysis_process.py", line 138, in _Main
    self._analysis_mediator.ProduceAnalysisReport(self._analysis_plugin)
  File "analysis/mediator.py", line 95, in ProduceAnalysisReport
    analysis_report = plugin.CompileReport(self)
  File "analysis/interface.py", line 272, in CompileReport
    self._LogProgressUpdateIfReasonable()
  File "analysis/interface.py", line 250, in _LogProgressUpdateIfReasonable
    completion_time = time.ctime(current_time + self.EstimateTimeRemaining())
  File "analysis/interface.py", line 312, in EstimateTimeRemaining
    estimated_seconds_per_batch = average_analysis_time + wait_time_per_batch
TypeError: can only concatenate tuple (not "int") to tuple

Traceback (most recent call last):
  File "plaso/multi_processing/analysis_process.py", line 138, in _Main
    self._analysis_mediator.ProduceAnalysisReport(self._analysis_plugin)
  File "plaso/analysis/mediator.py", line 95, in ProduceAnalysisReport
    analysis_report = plugin.CompileReport(self)
  File "plaso/analysis/interface.py", line 280, in CompileReport
    hash_analysis)
  File "plaso/analysis/interface.py", line 187, in _HandleHashAnalysis
    event_uuids = self._event_uuids_by_pathspec.pop(pathspec)
KeyError: <dfvfs.path.os_path_spec.OSPathSpec object at 0x7fc9e1e47e90>

[joachimmetz]

https://codereview.appspot.com/312850043/

[joachimmetz]

I've also create the following instructions: https://github.com/log2timeline/plaso/wiki/Analysis-plugin:-nsrlsvr

Feel free to improve them.

[pettai]

@joachimmetz Yes, I patched and built a nsrllookup that defaults to query localhost (As I don't wanna spam/leak my testing/debugging)

[joachimmetz]

changes merged