log/syslog: generated syslog format is not RFC-compliant

svent commented 7 months ago

Go version

go version go1.22.1 linux/amd64

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOVERSION='go1.22.1'

What did you do?

I was tracking down a broken syslog implementation in an application and was surprised that the log/syslog package produces a message format that is not compliant with any of the RFCs specifying valid syslog formats (RFC 3164 and RFC 5424).

Example Code: https://go.dev/play/p/cs5p3N3RZrJ

The format produced by the example code looks like this:

<28>2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

While full compliance with the RFCs is often not necessary, this format prevents even basic parsing of the timestamp and the sending hostname, as many implementations separate the fields by counting whitespaces (after detecting the syslog RFC variant being used).

RFC 3164 says:

The HEADER contains two fields called the TIMESTAMP and the HOSTNAME.
The TIMESTAMP will immediately follow the trailing ">"
The TIMESTAMP field is the local time and is in the format of "Mmm dd hh:mm:ss" (without the quote marks)

RFC 5424 defines:

SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG]
HEADER = PRI VERSION SP TIMESTAMP SP HOSTNAME SP - APP-NAME SP PROCID SP MSGID
PRI = "<" PRIVAL ">"
PRIVAL = 1*3DIGIT ; range 0 .. 191
VERSION = NONZERO-DIGIT 0*2DIGIT
TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME
"The TIMESTAMP field is a formalized timestamp derived from [RFC3339]."

The current implementation uses the structure of the old RFC 3164 (PRI followed immediately by the TIMESTAMP), but the timestamp is in the RFC3339 format used by the new RFC 5424.

The example message with a RFC 3164 header would look like:

<28>Nov 10 23:00:00 c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

The example message with a RFC 5424 header would look like:

<28>1 2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

I would suggest to support the new RFC 5424 by adding 1 (¨1" followed by a whitespace) after the PRI field. This should help many implementations parsing syslog messages.

If this is accepted, I would be willing to contribute the necessary code changes.

What did you see happen?

A syslog message in a format that is not compliant with RFC 3164 or RFC 5424:

<28>2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

What did you expect to see?

A syslog message with basic compliance with RFC 5424:

<28>1 2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

dmitshur commented 7 months ago

Thanks for reporting.

As a side note, the package comment states the log/syslog package is frozen and that some external packages provide more functionality.

CC @robpike.

robpike commented 7 months ago

I suggest using an external package instead. This package is frozen, as @dmitshur says.

mpx commented 7 months ago

https://go.dev/wiki/Frozen indicates a package "continues to be maintained, meaning regressions or breakages are fixed". Also: "It does not imply that the package should not be used."

This looks like more like a bug (rather than a feature request), where the package uses an incompatible format over the network?

When operating locally, it appears to use a valid RFC 3164 format:

timestamp := time.Now().Format(time.Stamp)
_, err := fmt.Fprintf(n.conn, "<%d>%s %s[%d]: %s%s",
    p, timestamp,
    tag, os.Getpid(), msg, nl)

...but uses a broken RFC 5424 format when Dialing a remote server:

timestamp := time.Now().Format(time.RFC3339)
_, err := fmt.Fprintf(n.conn, "<%d>%s %s %s[%d]: %s%s",
    p, timestamp, hostname,
    tag, os.Getpid(), msg, nl)

This seems like a simple fix for a bug. It is missing the Syslog VERSION (change the format to "<%d>1 %s %s %s[%d]: %s%s").

The package is still good for many simple use cases (I've used it locally). Seems reasonable to fix the network support in the frozen package?

Otherwise, perhaps the package should be documented as Deprecated with buggy network support?

Edit: Fixed RFC3164 code - copied wrong section originally

robpike commented 7 months ago

Those two snippets are identical.

mpx commented 7 months ago

Oops, sorry. Missed I copied the wrong code - fixed above.

golang / go