Timezone in logs

[llange]

I'm feeding the lmd logs to an external log parser, and having the timezone of the log entry would be a welcome addition.

On our systems, we use lmd embedded in Thruk. The server timezone is UTC, Thruk itself has a specific timezone in the configuration file. This has the resulting effect that :

• lmd log files have a time expressed in local time (the one configured in Thruk configuration file)
• while Thruk logs have a time expressed in UTC

Example of log entries differing only by a few seconds, local time is UTC+2:

[2021/04/28 14:38:02][thruk-dev][ERROR][thruk.log] FAILED - to load command module: logcache.
[2021-04-28 16:38:51.202][Info][listener.go:152] incoming status request from @ to /var/cache/thruk/lmd/live.sock finished in 99.844µs, response size: 196 B

(you can see the 2 hours (and few seconds) difference between the 2 lines, difference explained by the fact that one time is expressed in UTC, the other in UTC+2)

As there is no timezone specifier in lmd log files (nor in Thruk log files but it's another issue), it's not always easy to tell the external log parser what to do.

This issue is about adding:

• either, in a systematic way, the timezone to the logs (and, specifically, the timezone offset expressed like "-0700" without any separating colons) - by patching logging.go here (which may introduce compatibility issues for users already parsing the date/time)
• or having additional configuration option to specify the format for date/time in the logs

If this issues is accepted, and if you would like that I provide a PR, please provide guidance as which solution could be implemented, and if there are some conventions for the configuration parameters naming etc...

Thanks for your feedback.

[sni]

i'd say this is more a bug in Thruk then. Logfiles should never be written in the user selected timezone and only use the machines timezone setting or utc. When it comes to log parsing, i prefer the Grafana Loki way and simply "tail" the logfile and create the timestamp yourself on each log event.

[llange]

It may be a bug, I did not dig enough in Thruk to be sure of it.

Agreed that the tail way is interesting ; the one we use (Cloudwatch Agent) does it quite the same but I suspect it's a mix of polling and inotify - there can sometimes be a few seconds delay between the timestamp in the logs on the disk and the timestamp in the imported log entry.

Nevertheless, what about adding the timezone offset to the date (maybe in a configurable way) ? Even for a human, it unambiguous when reading logs in the past. (My own preference, regarding to timestamps, is to always have the timezone offset, even if logs are in UTC, to remove any ambiguity)

[sni]

closing this one. lmd knows nothing about timezones and only uses unix timestamps whenever timestamps are required.