log2timeline / plaso

Super timeline all the things
https://plaso.readthedocs.io
Apache License 2.0
1.73k stars 351 forks source link

Catch "too many open files" error #2193

Open Onager opened 6 years ago

Onager commented 6 years ago

Description of problem:

Users receive Python tracebacks when the run out of file handles. See: https://github.com/log2timeline/plaso/issues/1565 https://github.com/log2timeline/plaso/issues/669 https://github.com/log2timeline/plaso/issues/580

The typical advice we give is to increase the number of open file handles. This error is typically triggered when there are a large number of EWF files.

Command line and arguments:

N/A

Source data:

Any, but usually highly-split EWF files.

Plaso version:

HEAD

Operating system Plaso is running on:

Usually Windows or MacOS

Installation method:

N/A

Debug output/tracebacks:

[ERROR] (MainProcess) PID:8967 Unable to open file system with error: [Errno 24] Too many open files: 
Traceback (most recent call last):
File "/usr/local/bin/log2timeline.py", line 4, in 
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/init.py", line 742, in run_script
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/init.py", line 1667, in run_script
File "/Library/Python/2.7/site-packages/plaso-20171218-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 68, in 
File "/Library/Python/2.7/site-packages/plaso-20171218-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 54, in Main
File "/Library/Python/2.7/site-packages/plaso-20171218-py2.7.egg/plaso/cli/log2timeline_tool.py", line 572, in ExtractEventsFromSources
File "/Library/Python/2.7/site-packages/plaso-20171218-py2.7.egg/plaso/multi_processing/task_engine.py", line 734, in ProcessSources
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/init.py", line 123, in cpu_count
OSError: [Errno 24] Too many open files
2016-02-11 15:32:38,023 [WARNING] (MainProcess) PID:5984 <<string>> Unable to scan source with error: Unable to process source path specification with error: [Errno 24] Too many open files: u'e:\\grig-1000\\1tb-1-grig.EPS'.
Onager commented 6 years ago

The idea here is that rather than just crashing, we'd display a more helpful error message, then exit/crash.