search

CiscoSecurity / fp-05-firepower-cli

Public Repo for an eStreamer CLI project
10 stars 8 forks source link

event collector stops with TimeoutException error and does not restart or exit, running indefinitely until restarted manually #17

Open Koguni31 opened 1 year ago

Koguni31 commented 1 year ago

eNcore 5.2.4 runs and exports events for some time (from several or tens of minutes to tens of hours or several days) but stops with TimeoutException error and does not restart or exit, but several processes running indefinitely (and systemd does not see that service have failed) until restarted manually:

2023-05-03 08:38:06,308 Monitor INFO Running. 5609100 handled; average rate 2032.07 ev/sec; 2023-05-03 08:40:13,200 Monitor ERROR Monitor __start: 'TimeoutException' object has no attribute 'message' 2023-05-03 20:52:03,552 Controller INFO Stopping... 2023-05-03 20:54:04,070 Controller INFO eNcore version: development 2023-05-03 20:54:04,071 Controller INFO Python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] 2023-05-03 20:54:04,071 Controller INFO Platform version: Linux-5.15.0-69-generic-x86_64-with-glibc2.35 2023-05-03 20:54:04,071 Controller INFO Starting client (pid=2510035).

Before emitting error and stopping processing events, htop shows 100% vCPU usage by one of eNcore processes, this could be not related to problem, but anyway, its illustrated below:

Screenshot 2023-05-03 214735

Config file: estreamer.conf.txt

Systemd estreamer service file: estreamer.service.txt

Koguni31 commented 1 year ago

More failures with same behavior. Error messages in log:

2023-05-11 11:29:33,522 Controller INFO Process writer is dead. 2023-05-17 10:00:14,607 Monitor ERROR Monitor __start: 'BrokenPipeError' object has no attribute 'message' 2023-05-19 10:10:44,361 Controller INFO Process subscriber is dead.