Trying to concatenate Multi line records into single log record
Working well for all kinds of exceptions/startup logs
Lengthy exceptions are getting dropped.
...
Steps to replicate
Please tryout below config and seeing lengthy lines/exceptions are getting dropped.
Would like to know if we can limit lines or increase if its related to Buffer limitation
Provide example config and message
2020-04-03T02:57:17.605Z ERROR [Server] FATAL error: com.vmware.vdi.pop.configuration.ConfigurationException$IOConfigurationException: Failed to read configuration
com.vmware.vdi.pop.configuration.ConfigurationException$IOConfigurationException: Failed to read configuration
at com.vmware.vdi.pop.configuration.ConfigurationManager.read(ConfigurationManager.java:74)
at com.vmware.vdi.pop.configuration.ConfigUtils.readConfig(ConfigUtils.java:88)
at com.vmware.vdi.hydra.connector.server.Server.main(Server.java:287)
Caused by: java.io.FileNotFoundException: ./tls/connector-api.config (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:916)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3070)
at com.vmware.vdi.pop.configuration.ConfigurationManager.read(ConfigurationManager.java:68)
... 2 common frames omitted
Native Memory Tracking:
Total: reserved=2315290KB, committed=68470KB
concat plugin should be able concat above multiline exception/message into single log record. OR Need a way to truncate instead of dropping entire message.
...
Problem
Trying to concatenate Multi line records into single log record Working well for all kinds of exceptions/startup logs Lengthy exceptions are getting dropped. ...
Steps to replicate
Please tryout below config and seeing lengthy lines/exceptions are getting dropped. Would like to know if we can limit lines or increase if its related to Buffer limitation
Provide example config and message
2020-04-03T02:57:17.605Z ERROR [Server] FATAL error: com.vmware.vdi.pop.configuration.ConfigurationException$IOConfigurationException: Failed to read configuration
com.vmware.vdi.pop.configuration.ConfigurationException$IOConfigurationException: Failed to read configuration
at com.vmware.vdi.pop.configuration.ConfigurationManager.read(ConfigurationManager.java:74)
at com.vmware.vdi.pop.configuration.ConfigUtils.readConfig(ConfigUtils.java:88)
at com.vmware.vdi.hydra.connector.server.Server.main(Server.java:287)
Caused by: java.io.FileNotFoundException: ./tls/connector-api.config (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:916)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3070)
at com.vmware.vdi.pop.configuration.ConfigurationManager.read(ConfigurationManager.java:68)
... 2 common frames omitted
Native Memory Tracking:
Total: reserved=2315290KB, committed=68470KB
Expected Behavior
concat plugin should be able concat above multiline exception/message into single log record. OR Need a way to truncate instead of dropping entire message. ...
Your environment
Concat Plugin Config <filter **> @type concat key log multiline_start_regexp /^\d{4}[-/]\d{1,2}[-/]\d{1,2}/ separator "" flush_interval 5
OS version Linux dev1-westus2-jumpbox 5.0.0-1028-azure #30~18.04.1-Ubuntu SMP Fri Dec 6 11:47:59 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
paste result of
fluentd --version
ortd-agent --version
plugin version
fluent-gem list
,td-agent-gem list
or your Gemfile.lock fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch