Add function parse spring boot default log format

pengliaoye commented 1 year ago

A note for the community

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

example log format:

2023-01-30 22:37:33.495 INFO 72972 --- [ main] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel cacheConsumer-in-0 2023-01-30 22:37:33.514 INFO 72972 --- [ main] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel nullChannel 2023-01-30 22:37:33.520 INFO 72972 --- [ main] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel reqDataConsumer-in-0 2023-01-30 22:37:33.528 INFO 72972 --- [ main] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageHandler _org.springframework.integration.errorLogger

want add a parse_spring_boot function

Attempted Solutions

No response

Proposal

No response

References

No response

Version

No response

zamazan4ik commented 1 year ago

@pengliaoye do you have a link where Spring Boot log format is described? It helps a lot with the implementation of the parser.

pengliaoye commented 1 year ago

@zamazan4ik hi, this https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.logging.log-format https://rollbar.com/blog/spring-boot-logging/ https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml

and if exception log, the message will multiline

StephenWakely commented 1 year ago

This looks fairly straight forward. Have you tried using parse_grok for this?

pengliaoye commented 1 year ago

"(?m)%{TIMESTAMP_ISO8601:logtime}%{SPACE}%{LOGLEVEL:level}%{SPACE}%{POSINT:pid}%{SPACE}---%{SPACE}\[%{DATA:thread}\]%{SPACE}%{DATA:logger}%{SPACE}:%{SPACE}%{GREEDYDATA:message}"

this seems working.

jszwedko commented 1 year ago

https://github.com/vectordotdev/vrl/issues/4

ztkmkoo commented 1 year ago

Hi, anyone is assigned to this issue? If not I wanna try it. Thx

jszwedko commented 1 year ago

Hey!

We'd be happy to see a contribution here! You can look at this PR adding a different parsing function as an example: https://github.com/vectordotdev/vector/pull/6796

vectordotdev / vrl