fluent-plugins-nursery / fluent-plugin-systemd

This is a fluentd input plugin. It reads logs from the systemd journal.
Apache License 2.0
153 stars 43 forks source link

How can i use multiline ?? #59

Closed JooyoungJeong closed 6 years ago

JooyoungJeong commented 6 years ago

how can i use multiline ??


@test@ FATAL io.robustperception.java_examples.JavaSimple$ExampleServlet Fatal : You chose a number > 100 java.lang.NullPointerException: NullError at io.robustperception.java_examples.JavaSimple$ExampleServlet.doGet(JavaSimple.java:35) at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:648) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:365) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:926) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:988) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)

errm commented 6 years ago

This plugin does not support multiline... see https://github.com/reevoo/fluent-plugin-systemd/issues/11

If you are interested on working on this... I am happy to accept a PR... But I have no interest in implementing it myself :)

Jitsusama commented 6 years ago

@JooyoungJeong; multiline log handling can be handled before this plugin receives the event record. Common ways of handling this is via the concat filter plugin. I've used this myself with great success.

errm commented 6 years ago

I have not tried but it does seem that systemd might support log entries with newlines in them... If this works then this plugin should read them correctly without any further configuration ... I will try to make this work and report back

Jitsusama commented 6 years ago

@errm; that works for events sent to systemd as a single event, but more often than not, particularly in the case of Docker when it's configured to use systemd as its log-driver, they are sent to systemd as a separate event per log line. This is something I know all too well from my own personal experience.

errm commented 6 years ago

I guess the other option would be to configure your applications logger to escape the newlines ... This guy has some examples about doing it in Java https://fabianlee.org/2018/03/09/java-collapsing-multiline-stack-traces-into-a-single-log-event-using-spring-backed-by-logback-or-log4j2/

JooyoungJeong commented 6 years ago

Thank you for reply. The form of exception error is not fixed(each language). So I redefined log pattern in application. I use log4j2 and i defined pattern as follows.
"exception":"%replace{%xEx}{\n|\tat}{ }"

errm commented 6 years ago

Sounds good... I will mention these two strategies in the docs... It does seem where the application is using a logger that can be configured the best option is to replace the newlines in each message to keep them as one systemd entry ...