Closed magneticstain closed 6 years ago
Will need to refactor function itself, references to it, and the unit tests for it.
No exceptions during test runs and unit tests are passing. Should be good now.
2017-11-25 12:13:27,452 [ DEBUG ] [ lib.destiny.Sage ] network threat engine is sleeping for [ 15 ] seconds before restarting routines
2017-11-25 12:13:42,465 [ INFO ] [ lib.destiny.Sage ] fetching intel, baseline, and log data (in that order) for threat detection model
2017-11-25 12:13:42,465 [ DEBUG ] [ lib.destiny.Sage ] fetching training intel data
2017-11-25 12:13:42,465 [ DEBUG ] [ lib.destiny.Sage ] fetching logs from log db
2017-11-25 12:13:42,465 [ DEBUG ] [ lib.destiny.Sage ] decoding all log data from log db
2017-11-25 12:13:44,067 [ DEBUG ] [ lib.destiny.Sage ] fetched and decoded [ 100 ] log records from db
2017-11-25 12:13:44,067 [ DEBUG ] [ lib.destiny.Sage ] fetching training baseline data
2017-11-25 12:13:44,067 [ DEBUG ] [ lib.destiny.Sage ] fetching logs from log db
2017-11-25 12:13:44,067 [ DEBUG ] [ lib.destiny.Sage ] decoding all log data from log db
2017-11-25 12:13:56,348 [ DEBUG ] [ lib.destiny.Sage ] fetched and decoded [ 71542 ] log records from db
2017-11-25 12:13:56,445 [ DEBUG ] [ lib.destiny.Sage ] fetching testing (raw log) data
2017-11-25 12:13:56,446 [ DEBUG ] [ lib.destiny.Sage ] fetching logs from log db
2017-11-25 12:13:56,446 [ DEBUG ] [ lib.destiny.Sage ] decoding all log data from log db
2017-11-25 12:14:03,971 [ DEBUG ] [ lib.destiny.Sage ] fetched and decoded [ 41270 ] log records from db
2017-11-25 12:14:04,011 [ DEBUG ] [ lib.destiny.Sage ] initializing intel data
2017-11-25 12:14:04,024 [ DEBUG ] [ lib.destiny.Sage ] calculating unique field list for training and testing data initialization
2017-11-25 12:14:04,216 [ DEBUG ] [ lib.destiny.Sage ] initializing log data matrix
2017-11-25 12:14:04,730 [ INFO ] [ lib.destiny.Sage ] vectorizing log data
2017-11-25 12:14:06,997 [ INFO ] [ lib.destiny.Sage ] training threat detection model
2017-11-25 12:14:08,252 [ INFO ] [ lib.destiny.Sage ] training complete; starting network threat analysis against current log data
2017-11-25 12:14:08,252 [ DEBUG ] [ lib.destiny.Sage ] initializing log data
2017-11-25 12:14:08,252 [ DEBUG ] [ lib.destiny.Sage ] initializing log data matrix
2017-11-25 12:14:08,632 [ INFO ] [ lib.destiny.Sage ] vectorizing log data
2017-11-25 12:14:09,521 [ INFO ] [ lib.destiny.Sage ] making predictions for testing data
2017-11-25 12:14:10,126 [ INFO ] [ lib.destiny.Sage ] threat detection results :: { [0 0 0 ..., 0 0 0] }
2017-11-25 12:14:10,126 [ DEBUG ] [ lib.destiny.Sage ] processing threat detection results
2017-11-25 12:14:10,128 [ DEBUG ] [ lib.destiny.Sage ] network threat engine is sleeping for [ 15 ] seconds before restarting routines
josh@jimmy1 ~/Code/Inquisition $ python3 -m unittest discover -s build/tests/ && python3 -m unittest discover -s build/test_TESTS/
...............................................Exception ignored in: <bound method Pygtail.__del__ of <pygtail.core.Pygtail object at 0x7fa365367ac8>>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/pygtail/core.py", line 91, in __del__
if self._filehandle():
File "/usr/local/lib/python3.5/dist-packages/pygtail/core.py", line 173, in _filehandle
self._fh = open(filename, "r", 1)
PermissionError: [Errno 13] Permission denied: '/var/log/inaccessible_test_log'
.Exception ignored in: <bound method Pygtail.__del__ of <pygtail.core.Pygtail object at 0x7fa3630dca20>>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/pygtail/core.py", line 91, in __del__
if self._filehandle():
File "/usr/local/lib/python3.5/dist-packages/pygtail/core.py", line 173, in _filehandle
self._fh = open(filename, "r", 1)
FileNotFoundError: [Errno 2] No such file or directory: '/var/log/non_existant_file'
....................
----------------------------------------------------------------------
Ran 68 tests in 36.582s
OK
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
josh@jimmy1 ~/Code/Inquisition $
This is a Codacy recommendation and will make the function more clear to devs.