vectordotdev / vrl

Vector Remap Language
Mozilla Public License 2.0
123 stars 56 forks source link

`parse_syslog` incorrectly adds the previous year for RFC3464 timestamps when close to the new year and the system timezone is not UTC #641

Open yangshike opened 7 months ago

yangshike commented 7 months ago

A note for the community

Problem

When the system time is (+800) 2024-01-01 00:00:00 - 2024-01-01 08:00

The timestamp field parsed through parse_syslog becomes 2022, this bug is reproducible

screenshot:

image image

Configuration

[sources.message_log]
type = "file"
include = ["/var/log/messages"]
start_at_beginning = true
[transforms.modify_message_log]
type = "remap"
inputs = [ "message_log" ]
source = '''
. = parse_syslog!(.message)
'''
[sinks.console]
type = "console"
inputs = ["modify_message_log"]
encoding.codec = "json"

Version

0.33

Debug Output

No response

Example Data

Comparison of original log and log processed by vector: image

Additional Context

No response

References

No response

drmason13 commented 5 months ago

The year is inferred using the current time in UTC, and the month as parsed from the log message (without any conversion).

This seems to be where the problems occur. The message's month could be different in UTC if the message is in a different timezone. The code is shared between Vector and a dependency syslog_loose.

I don't understand what's gone wrong exactly and how it managed to end up with 2022, but I think it's likely to be in this resolve_year and syslog_loose interaction.

Timezones are hard.