xcrezd
commented
2 years ago Same issue here
Closed shalousun closed 1 year ago
xcrezd
commented
2 years ago Same issue here
RicardoAAD
commented
1 year ago Hello @shalousun
Thanks for bringing this issue to our attention. I did a quick test using the log you provided, and it is easily reproducible even with a newer version of Fluen-Bit.
I checked the java built-in multiline parser, which is working as expected for Google Cloud Java language applications. Unfortunately, it doesn't work with the log example you provided. The built-in java multiline parser uses rules to specify how to match a multiline pattern and perform the concatenation. Each rule has a regex to determine if a line is either the start of a multiline log or if it is part of the stack trace log.
The regex of the Java multiline parser searches for the words "Exception" | "Error" | "Throwable" | "V8 errors stack trace" and when it matches any of these words, Fluent-Bit sets this line as the start of a multiline log. It will concatenate everything that matches with the regex in the other rules that define the rest of the java stack trace. This is the reason why Fluen-Bit shows two lines after processing the log sample that you have provided.
In line0 you have 2022-07-31 23:41:34.668 [http-nio-8080-exec-10] ERROR com.benchmark.springboot.error.RestExceptionHandler@unknownException:77 - Error code 500：{}
and in line1 you have java.lang.ArithmeticException: / by zero
When line1 is read, Fluent-Bit sends the line0 to the configured output, and sets line1 as a new start of a multiline log, then concatenates the rest of the stack trace lines to line1.
We wanted to have more details about the code triggering these Exceptions; this is why we wanted to ask for a repo sample of the project/code where you see these Exceptions, so we can study your repro and analyze this issue in detail. As workaround you can use these.
You can use these custom multiline parser that works with the log example you provided. Please check this doc about the custom multiline parsers Config used
[INPUT]
name tail
path /tmp/test.log
read_from_head true
multiline.parser multiline_java
tag multiline
[OUTPUT]
name stdout
match *
[SERVICE]
Flush 1
Daemon Off
Log_Level info
parsers_file multiline_parser.confmultiline_parser.conf:
[MULTILINE_PARSER]
name multiline_java
type regex
key_content log
flush_timeout 2000
trace_error on
# rules | state name | regex pattern | next state
# --------|----------------|---------------------------------------------
rule "start_state" "/^(\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}.*)/" "cont"
rule "cont" "/^([a-zA-Z.]+Exception):\s(.*)$/" "cont"
rule "cont" "/^\s+at.*/" "cont"Please let us know if the custom multiline example works for you or if you need additional help to tune this example to any other use case you may have.
Regards, Ricardo
github-actions[bot]
commented
1 year ago This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.
github-actions[bot]
commented
1 year ago This issue was closed because it has been stalled for 5 days with no activity.
Bug Report
Describe the bug Handling java exception log errors using multiline filter，A complete exception log is split into two，The configuration is as follows
after filter, first logs content is
second log content
To Reproduce