alternative log rotation strategy that avoids truncating logs

[PanieriLorenzo]

Currently logs are truncated when they are rotated. This is how it is usually done in the Linux world, but it makes it hard to use log aggregation systems such as LogStash or FileBeat.

To facilitate the use of such tools, it would be nice if it was possible to change the log rotation strategy, such that the names of log files are always unique, and logs are append-only, with no truncation. For example, the name of the files can be based on the date, and at midnight, NextCloud just starts logging to a new file with the next day's date.

Currently I have set up a cron job that changes the name of log files in config.php once a day at midnight, but this is pretty ugly.

For context, I'm running NextCloud in Docker, and I need to use FileBeat to collect the log files into a database.

[kesselb]

Sounds like a good enhancement.

Would logging to stdout also work?

[PanieriLorenzo]

I suppose it would, FileBeat has a module specifically for reading Docker container stdout and stderr logs. But since I am collecting multiple log files (nextcloud and audit) it would be nice to have them in dedicated files.

[joshtrichards]

@PanieriLorenzo - Couple thoughts:

• You're not locked into using Nextcloud's built-in log rotation: use the logrotate tool which offers far more configuration including managing timestamped filenaming/etc (it's the same tool used to manage file-based log files in most Linux distributions already) (e.g. [1])
• We don't copy+truncate the logs; we just rename the existing active log (nextcloud.log) to nextcloud.log.1[2]
• From reading Filebeat's docs, it should support how we do things currently already[3]
• We also support syslog; you're not locked into file-based logs. Filebeat supports syslog and in that case logs go directly there so rotation is no longer relevant

[1] https://linux.die.net/man/8/logrotate [2] https://github.com/nextcloud/server/blob/a64df1908bb10b103ce98235ff21d0087d6978d7/lib/public/Log/RotationTrait.php#L49-L53 [3] https://www.elastic.co/guide/en/beats/filebeat/current/file-log-rotation.html