search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
109 stars 3.51k forks source link

RFE: logrotation to logstash docker container #10132

Open fenchu opened 6 years ago

fenchu commented 6 years ago

Hi

https://github.com/elastic/logstash/blob/master/Dockerfile (v6.4.2)

If you have a lot of activity in logstash, the logfiles can grow very large and fill up entire disk. This is the default stdout log which you access when doing docker logs not log4j2.conf

We have recently seen the docker host running out of disk very frequently until we limited it at runtime with --log-opt max-size=100m --log-opt max-file=10 for a max of 1GB log

# find / -xdev -type f -size +50M -exec du -sh {} ';' | sort -rh
14G     /var/lib/docker/containers/96182894b9f2ab1bce254a64643b7b5e7682aa13ca125d7edbe989746cefdb52/96182894b9f2ab1bce254a64643b7b5e7682aa13ca125d7edbe989746cefdb52-json.log

Please see: https://docs.docker.com/config/containers/logging/json-file/#options

praseodym commented 5 years ago

This is something you’ll have to fix in your own environment. Logstash simply logs to stdout, and by default Docker will keep this output indefinitely. Setting the max-size and max-file parameters or using another Docker logging driver are possible solutions, and both are independent from the Logstash container image itself.