scalyr / scalyr-agent-2

The source code for Scalyr Agent 2, the daemon process Scalyr customers run on their servers to collect metrics and logs.
Apache License 2.0
70 stars 60 forks source link

Syslog memory usage fix #1270

Closed alesnovak-s1 closed 3 months ago

alesnovak-s1 commented 3 months ago

After several attempts to make syslog monitor threading robust and memory efficient, I decided to implement a simple solution of bounded ThreadingMixin. A better candidate might seem to be a ThreadPoolExecutor, but it proved to be very memory hungry.

The reason for using threads in the first place is:

The ideal solution would be to get right of our rigid in-house implementation based on BaseServer and etither

github-actions[bot] commented 3 months ago

Test Results

     20 files  ±0       20 suites  ±0   30m 40s :stopwatch: - 1m 12s 1 489 tests +1  1 469 :heavy_check_mark: +1    20 :zzz: ±0  0 :x: ±0  7 116 runs  +5  6 889 :heavy_check_mark: +3  227 :zzz: +2  0 :x: ±0 

Results for commit d9ac1355. ± Comparison against base commit 2abc1ca2.

:recycle: This comment has been updated with latest results.