search

GregYankovoy / docker-goaccess

goaccess reverse proxied through nginx for ease of use
MIT License
32 stars 14 forks source link

Handle logrotate files? #3

Closed EpochalEngineer closed 4 years ago

EpochalEngineer commented 4 years ago

Is it possible to add handling of logrotated gzipped files with this?

https://github.com/allinurl/goaccess/issues/1406

Would be an alternate solution to https://github.com/GregYankovoy/docker-goaccess/issues/2

GregYankovoy commented 4 years ago

Hi,

The solution you have referenced will include any files that have been previously gzipped. It will not pick up files that are gzipped after the container has been launched.

I am thinking it may be best to keep an external container/cronjob/script responsible for unzipping the files into the monitored log directory. The goaccess container can then pick up the log files without any modifications to it's current implementation.

One thing to keep in mind is that goaccess implementation of filesystem/database storage does not work in docker. So any analyzed logs will take up RAM. See previous thread detailing this issue

Let me know if this helps, and if not we can talk more about some solutions.

Thanks, Greg

EpochalEngineer commented 4 years ago

Yeah, that sounds like a better solution. I'd like to maintain real-time monitoring with all of the history, not take up too much RAM or disk space, and not regularly re-extract logfiles.

Seems like goaccess won't handle wildcards in the .config file anyway. So a decent solution is to have a separate bind mount for the real-time log, and another script that checks the number of files, re-extracting when that changes to an archived log, and watch both with go-access. And store that on a compressed filesystem.