search

graylog-labs / graylog2-web-interface

[DEPRECATED]
https://www.graylog.org/
611 stars 174 forks source link

Syslog messages getting split up in chunks. #1709

Closed chrisjohansen closed 8 years ago

chrisjohansen commented 8 years ago

I have a setup with SolrCloud nodes that are sending log4j messages to our graylog server. Many of those messages are rather large. Graylog2 recieves the Whole message but it is being split up in chunks.

Chunk 1 - * "Message bla bla bla..." Chunk 2 - * "... bla bla bla bla bla..." Chunk 3 - * "... bla bla bla."

Does anyone have an idea how to avoid this. I want 1 complete message. I tried changing the maxsize on graylog2 server and recived buffer size but none seems to Work.

Input Config: Syslog_UDP (Syslog UDP) recv_buffer_size: 4194304 port: 10514 override_source: force_rdns: true allow_override_date: true bind_address: x.x.x.x store_full_message: true

joschi commented 8 years ago

@chrisjohansen Syslog is line-oriented. Are you sure that SolrCloud (or rather the syslog client) isn't splitting the log messages and sending multiple syslog messages?

This being said, you could also simply use one of the many GELF appenders (see https://marketplace.graylog.org/addons?search=java) to send rich, structured log messages to Graylog directly. See https://cwiki.apache.org/confluence/display/solr/Configuring+Logging for some details about how to configure logging in SolrCloud.

chrisjohansen commented 8 years ago

Log4J is creating a local log file under /var/log/... and is looking fine. Logs are also piped to our external graylog2 server. All logs on client looks fine though just not when it hits Graylog2.

I will try and look into your suggestions regarding the log4j handler.

Thank you.