search

fluent-plugins-nursery / fluent-plugin-concat

Fluentd Filter plugin to concatenate multiline log separated in multiple events.
MIT License
108 stars 33 forks source link

Concat problem.... #62

Open omoldes opened 5 years ago

omoldes commented 5 years ago

Problem

This is a message:

2018-11-21 15:41:39,418 ERROR [controllers.FALoggingController:postLog:100] Error logging
org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [insert into falog (comment, data, deviceId, level, message, tag, timestamp, uploadId, userId, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; nested exception is org.hibernate.exception.DataException: could not execute batch
    at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:282)
    at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:244)
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:521)
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
    at 
Caused by: org.hibernate.exception.DataException: could not execute batch
    org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:110)
    ... 76 more
Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2125)
    ... 80 more
2018-11-21 15:41:58,578 INFO  [works.FAWorksManager:getWorks:251] 90046571 -> Download works from server: 0 documents and 0 media details

And my config for td-agent:

<filter apache.flexiapp.pa>
    @type concat
    key message
    key_name message
    stream_identity_key container_id

    multiline_start_regexp /^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3} .*/
    continue_line_regexp /^\s+at.*/
    continue_line_regexp /^\t+at.*/
    continue_line_regexp /^Caused by: .*/
    #continue_line_regexp /^... [0-9]+ more/
    #multiline_end_regexp /^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}/
    multiline_end_regexp /^... [0-9]+ more/

</filter>
okkez commented 5 years ago

You can use one continuous_line_regexp in the filter section.

continuous_line_regexp /^(\s+at|Caused by:)/
okkez commented 5 years ago

@omoldes Any problem?