multi-line logs (i.e. stack trace)

[thedug]

When logging stack traces or messages with \n character the lines are appearing in loggly as separate logs.

I'm considering switching my logs to json which would solve this problem, but before doing that I thought I would see if anybody has another solution.

[progrium]

Any stdout or stderr can only reasonably be parsed with newlines. Until logspout supports a syslog intake, JSON or other structured data that encodes newlines is the only option.

[ujjwal]

I am facing the same issue. Unfortunately the JSON logs don't contain the container_name -- the filename isn't useful either since it is the container_id.

Any known workarounds for that?

[progrium]

No not yet. Hoping to get a syslog port exposed so containers with more complex logs can use that similar to a system syslog, except it is container aware and uses the logspout routes.

On Mon, Jun 15, 2015 at 12:03 PM, Ujj notifications@github.com wrote:

I am facing the same issue. Unfortunately the JSON logs don't contain the container_name -- the filename isn't useful either since it is the container_id.

Any known workarounds for that?

— Reply to this email directly or view it on GitHub https://github.com/gliderlabs/logspout/issues/81#issuecomment-112139964.

Jeff Lindsay http://progrium.com

[brendangibat]

Not sure about loggly but for me I'm pumping my logs to Logstash and processing with a multiline filter that matches surrounding logs on the type of log we're processing.

[chrisferry]

Any status update on this?

[yqguodataman]

@thedug logspout how switching my logs to json?

[blackside]

So, any progress on this? Im sending my logs to a splunk server and Im trying to play around with a multiline filter but it would be sooo much easier if logspout had some awerensee of multiline events.

[progrium]

Like I said it needs a syslog server. If anybody wants to help add this then there would be progress.

On Tuesday, October 18, 2016, Carl notifications@github.com wrote:

So, any progress on this? Im sending my logs to a splunk server and Im trying to play around with a multiline filter but it would be sooo much easier if logspout had some awerensee of multiline events.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gliderlabs/logspout/issues/81#issuecomment-254476143, or mute the thread https://github.com/notifications/unsubscribe-auth/AAACh7lc1fYd0sWg_hz8q9_tDOMupl9Tks5q1KhFgaJpZM4D9v1p .

Jeff Lindsay http://progrium.com

[sybrandy]

Hello,

I'm a similar boat except I only really care about capturing stack traces if the program crashes. In our case, all of our generated logs, at least on the apps we control, will be formatted in a specific manner. However, if the program crashes, such as a panic in go, we have no control over the formatting of the stack trace.

So, this got me thinking about how we could handle this type of case without adding in syslog. What if we add an option to say that if a line starts with whitespace, it is combined with the previous line. I don't think this would be that hard to implement. Pseudo-code follows:

Read a line
If we are buffering:
    While next line starts with whitespace:
        Read next line
        Append new line to existing line(s)
Send message

The bufio package has a Peek method on the Reader type, so that can be used to do the peeking. If an error is returned, that means there is no new data at this time, so the message should be sent.

Thoughts? I think this may be a good solution for the simple case.

[ezh]

Is it possible to implement it like in a firebeat? https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

Reasons:

• it is already work
• it is simple (to debug too)
• it is opensource
• it is go

And it is simple to override per container - environment variable/tuple like "boolean,boolean,regex" or three docker labels

[progrium]

Yeah this seems reasonable.

On Fri, Feb 24, 2017 at 11:39 AM, Alexey Aksenov notifications@github.com wrote:

Is it possible to implement it like in a firebeat? https://www.elastic.co/guide/en/beats/filebeat/current/ multiline-examples.html

Reasons:

• it is already work
• it is simple (to debug too)
• it is opensource
• it is go

And it is simple to override per container - environment variable/tuple like "boolean,boolean,regex" or three docker labels

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gliderlabs/logspout/issues/81#issuecomment-282354375, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAChzvYsB6cLRYi9LY0KI6ZHlGyfH7Pks5rfxXRgaJpZM4D9v1p .

-- Jeff Lindsay http://progrium.com

[hugodlg]

Any news?

[josegonzalez]

If we had news or updates, they would be posted in this issue. We're aware of the requirement some developers have, but as this is an open source project, a solution will be available once someone contributes it.

If you have a pull request implementing this functionality, we'd be happy to review it. Otherwise, please hold off on asking for updates, as there aren't any.

[lukasmrtvy]

https://github.com/anashaka/logspout-logstash/blob/master/multiline/multiline.go

[masterada]

Added pull request, please review & recommend changes.