newrelic / newrelic-fluent-bit-output

A Fluent Bit output plugin that sends logs to New Relic
Apache License 2.0
26 stars 42 forks source link

Multi line logs not working #115

Closed marcelocastrillo closed 4 months ago

marcelocastrillo commented 2 years ago

I have configured log fordwards to support multiline logs, but the logs are not being sent to new relic. If I do it the regular way, with logs.yml the logs are sent but one log per line.

This is the configuration:

logs.yml :

logs:

fluentbit.conf:

[INPUT] Name tail Path /home/tomcat/apache-tomcat-8.5.72/bin/test.log Path_Key filePath Key message Tag multiline_logs Multiline On Parser_Firstline MULTILINE_MATCH

parsers.conf

[PARSER] Name MULTILINE_MATCH Format regex Regex /(?[\d-]+ [\d:,]+) (?:[\w+] )?(?[A-Z]+)(?[\s\S]*)/

marcelschlapfer commented 2 years ago

Can you provide a sample of what the log messages look like? My assumption is that the regex probably doesn't match the message format correctly.

marcelocastrillo commented 1 year ago

I have the same configuration in another server, where the same app is running (with the same log4j configuration) and it works. The only difference is that the server where it works is running on debian, and this one is running on Centos ¿could that be a problem?

21/10/2022 01:25:10 ERROR en Clase: ImpCompras - MSN: could not extract ResultSet
 Exception.Class: GENERIC JDBC EXCEPTION
 Cause:org.postgresql.util.PSQLException: ERROR: Esta funcionalidad se encuentra temporalmente deshabilitada, por favor intente de nuevo mas tarde.
 ErrorCode: 0
 SQLException:org.postgresql.util.PSQLException: ERROR: Esta funcionalidad se encuentra temporalmente deshabilitada, por favor intente de nuevo mas tarde.
  Where: PL/pgSQL function f_get_tmp_det_necesidad_compra(date,date,bigint,bigint,bigint,character varying,character varying,bigint,bigint,bigint,bigint,bigint) line 9 at RAISE
 LastSQL:n/a
 StackTrace:
    at ar.com.dux.persistance.storedprocedures.implementations.ImpCompras.selectTmpDetNecesidadCompra(ImpCompras.java:92)
    at ar.com.dux.business.compras.implementations.ImpBusDetNecesidadCompra.selectTmpDetNecesidadCompra(ImpBusDetNecesidadCompra.java:41)
    at ar.com.dux.delegators.Compras.selectTmpDetNecesidadCompra(Compras.java:1060)
    at ar.com.dux.beans.compras.necesidadCompra.BBConsultaNecesidadCompraPorSucursal.actBtnConsultar(BBConsultaNecesidadCompraPorSucursal.java:93)
    at ar.com.dux.beans.compras.necesidadCompra.BBConsultaNecesidadCompraPorSucursal.<init>(BBConsultaNecesidadCompraPorSucursal.java:56)
    at ar.com.dux.jsf.filters.CacheFilter.doFilter(CacheFilter.java:27)
    at ar.com.dux.jsf.filters.SecurityFilter.conresultado(SecurityFilter.java:98)
    at ar.com.dux.jsf.filters.SecurityFilter.doFilter(SecurityFilter.java:83)
marcelschlapfer commented 1 year ago

What version of CentOS?

Also, just want to confirm that your regex looks like this: /(?<timestamp>[\d-]+ [\d:,]+) (?:\[\w+]\ )?(?<level>[A-Z]+)(?<message>[\s\S]*)/

marcelocastrillo commented 1 year ago

Centos version is 7. Is the regex incorrect? Because it is working on other servers.

marcelschlapfer commented 1 year ago

I tested your configuration on an Amazon Linux server and it works fine. CentOS 7 is supported by our Infra Agent. What version of the agent are you running? You could maybe also try with latest stand alone FluentBit agent to validate.

danybmx commented 4 months ago

Closing since no comments from Oct, 2022