search

Trippik / PfSense_Dashboard-Data_Collection_Server

Syslog server component of the PfSense Monitoring dashboard, formatted as a docker container it acts as a Syslog server processing incoming data from PfSense instances and storing them in an underlying MySQL database used by all containers making up the PfSense Dashboard
MIT License
0 stars 0 forks source link

Improved Frontend and System Performance #3

Open Trippik opened 1 year ago

Trippik commented 1 year ago

Investigate using PyPy for data collection for improved performance, and integration of DB caching

Trippik commented 1 year ago

Looking into moving to MariaDB for the underlying DB and implementing query caching to improve frontend performance with multiple users (https://mariadb.com/kb/en/query-cache/)

Trippik commented 1 year ago

Run into compatibility issues with SciPy (a required dependency) and PyPy (https://github.com/scipy/scipy/issues/16737). Will investigate cpython as an alternative

Trippik commented 9 months ago

This along with the auto-refresh could be implemented via using javascript in the frontend, to offload some processing to client machines

Trippik commented 3 months ago

Multithreading could help improve system performance for data_collection here to allow simultaneous log pulls from all clients, here to allow simultaneous processing of all traffic log messages)