Multi-line messages in logs

skanga commented 9 months ago

Sometimes in my log files I will have a multi-line stack trace. Is there a way to tell log-analyzer that if current line does NOT match the normal pattern then add it to the LAST field of the previous line?

Here is an example:

0xc<Mon Nov 20 04:53:45 UTC 2023> DEBUG <Resource> <ErrorRef: 34urn7qaqacaryracfb6wu2etm> Failed to get value [serverInfo] of mbean [Catalina:type=Server] 
0xc<Mon Nov 20 04:53:45 UTC 2023> DEBUG <Resource> <ErrorRef: nt22aaibjwgw4wzmlqwvxocgnq> Failed to invoke a method [getVersion] from a class [org.eclipse.jetty.server.Server]  <java.lang.ClassNotFoundException: org.eclipse.jetty.server.Server>
java.lang.ClassNotFoundException: org.eclipse.jetty.server.Server
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)

tom-draper commented 9 months ago

Hi, thanks for this suggestion! I've been meaning to add multi-line patterns for a while, which should allow for capturing these stack traces. I'll have an update on this very soon!

tom-draper commented 9 months ago

Multi-line patterns can now be used in config by including newline characters within the pattern string.

{
  "tokens": ["timestamp", "log_type", "error_ref", "error_msg", "exception", "location", "function", "file_name", "line_number", "function2", "file_name2", "line_number2"],
  "patterns": [
    "0xc<timestamp> log_type <Resource> <ErrorRef: error_ref> error_msg",
    "0xc<timestamp> log_type <Resource> <ErrorRef: error_ref> error_msg\nexception: location\n  at function(file_name:line_number)\n    at function2(file_name2:line_number2)",
  ]
}

Although, unless your stack traces have a fairly consistent number of lines, these patterns may be painful to build. I will have a look into the best way of also implementing something similar to your original suggestion.

tom-draper / log-analyzer

Multi-line messages in logs #1