Support for cloudfoundry/syslog-release log messages

[keymon]

We found the usage of the metron agent to configure rsyslog a little bit of "hackery".

It makes sense to keep this magic wrapped in a specific job, and that is why it makes sense to use https://github.com/cloudfoundry/syslog-release

But the metron_agent syslog message and the syslog-release message are not equivalent.

The logsearch-for-cloudfoundry filters are designed to use the metron_agent format only, and fail when using the syslog release. Also note that the syslog-release format does not include the job index.

There are a pair of issues in the loggregator/metron_agent and syslog_release to get then using the same standard rfc5424

• https://github.com/cloudfoundry/syslog-release/issues/3
• https://github.com/cloudfoundry/loggregator/issues/134

I open this issue to aggregate all the info. All the solutions require some coordination:

• Option 1: logsearch-for-cloudfoundry supports all formats
• Option 2: Adapt both syslog-release, metron_agent and logsearch-for-cloudfoundry to metron_agent format.
• Option 3: make syslog-release send logs as the metron_agent does.

[hannayurkevich]

Hi @keymon,

Thanks for spotting this problem.

I think that it makes sense to do changes for Option1 first. So that logsearch-for-cloudfoundry supports all formats.

Then, if it is decided to apply Option2 or Option3 and a unified format is used for metron_agent and syslog_release, then we will change logsearch-for-cloudfoundry to use this format.

I think you should address Option2 and 3 to authors/contributors of metron_agent and syslog_release. Meanwhile I'm going to update logsearch-for-cloudfoundry to use Option1 soon.

Hanna

[Infra-Red]

Fixed in https://github.com/cloudfoundry-community/logsearch-for-cloudfoundry/pull/207