RFC 5424 protocol support

[nickbreen]

RFC 3164 protocol is supported, which makes this a minor issue.

logger --server syslog-cloudwatch-bridge --port 514 --stderr --rfc3164 test message
<13>Nov  8 08:53:56 wp-1 root: test message

A fatal error occurs when using RFC 5424 protocol.

logger --server syslog-cloudwatch-bridge --port 514 --stderr --rfc5424 test message
<13>1 2016-11-08T08:54:26.921856+00:00 wp-1 root - - [timeQuality tzKnown="1" isSynced="0"] test  message

The following error is logged and the container exits with exit code 2.

2016/11/08 08:52:37 Starting syslog server on 0.0.0.0:514
2016/11/08 08:52:37 Logging to group: REDACTED
2016/11/08 08:52:37 Created CloudWatch Logs stream: REDACTED
panic: interface conversion: interface is nil, not string

goroutine 15 [running]:
panic(0x82e220, 0xc8202f5800)
    /usr/local/Cellar/go/1.6.2/libexec/src/runtime/panic.go:481 +0x3e6
main.sendToCloudWatch(0xc82032eed0)
    /Users/rjocoleman/.go/src/github.com/rjocoleman/syslog-cloudwatch-bridge/main.go:78 +0x2c0
main.main.func1(0xc820315800)
    /Users/rjocoleman/.go/src/github.com/rjocoleman/syslog-cloudwatch-bridge/main.go:62 +0x64
created by main.main
    /Users/rjocoleman/.go/src/github.com/rjocoleman/syslog-cloudwatch-bridge/main.go:64 +0x71c

[nickbreen]

To configure syslog-ng to forward using the RFC 3164 protocol use network (https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/reference-destination-network-chapter.html), e.g.:

  destination d_network_cloudwatch {
        network("syslog-cloudwatch-bridge" port(514));
  };

  log {
        source(s_src); destination(d_network_cloudwatch);
  };

Where syslog-cloudwatch-bridge is the container 'hostname'.

[rjocoleman]

It looks like the RFC5424 log parts are a different schema within the data structure when parsed.

https://github.com/rjocoleman/syslog-cloudwatch-bridge/blob/master/main.go#L78

https://github.com/jeromer/syslogparser provides a nice breakdown between the two formats

[nickbreen]

I'm not familiar with go, so can't suggest a code change.

Could the error be trapped so that it doesn't kill the process?

[nickbreen]

As far as I can read the go sources and the github.com/mcuadros/go-syslog dependencies, the RFC 3164 parser's LogParts has content which is referenced on https://github.com/rjocoleman/syslog-cloudwatch-bridge/blob/master/main.go#L78, but for RFC 5424 the equivalent field is message.

https://github.com/rjocoleman/syslog-cloudwatch-bridge/blob/master/main.go#L53 defines an Automatic parser, which will handle messages in any incoming format, but https://github.com/rjocoleman/syslog-cloudwatch-bridge/blob/master/main.go#L78 assumes content without checking what the message was parsed as.

I think parserWrapper obscures which parser was used, which leaves introspection of the LogParts to determine what protocol is in use.

To avoid the fatal error and killing the process https://github.com/rjocoleman/syslog-cloudwatch-bridge/blob/master/main.go#L75-L84 could be modified such:

if msg, rfc3164 := logPart["content"].(string); rfc3164 {
    params := &cloudwatchlogs.PutLogEventsInput{
        LogEvents: []*cloudwatchlogs.InputLogEvent{
            {
                Message:   aws.String(msg),
                Timestamp: aws.Int64(makeMilliTimestamp(logPart["timestamp"].(time.Time))),
            },
        },
        LogGroupName:  aws.String(logGroupName),
        LogStreamName: aws.String(streamName),
    }

} else if msg, rfc5424 := logPart["message"].(string); rfc5424 {
    params := &cloudwatchlogs.PutLogEventsInput{
        LogEvents: []*cloudwatchlogs.InputLogEvent{
            {
                Message:   aws.String(msg),
                Timestamp: aws.Int64(makeMilliTimestamp(logPart["timestamp"].(time.Time))),
            },
        },
        LogGroupName:  aws.String(logGroupName),
        LogStreamName: aws.String(streamName),
    }
} else {
    // log that the message format is unsupported?
}

... assuming that's valid go and that timestamp is common between protocols.