andreaconsadori
commented
9 years ago Closed andreaconsadori closed 9 years ago
andreaconsadori
commented
9 years ago 
joschi
commented
9 years ago The garbage in the log messages looks very suspicious. Could you please elaborate a little bit on your Graylog setup? Which version of Graylog are you running, how did you install it (manually, OS packages, virtual appliance…), which Java version are you using, and which operating system is Graylog running on?
andreaconsadori
commented
9 years ago Hi,it's a Graylog 1.0.0 (Stable) manually installed on 3.2.0-4-686-pae #1 SMP Debian 3.2.63-2+deb7u1 i686 GNU/Linux java version "1.7.0_72"Java(TM) SE Runtime Environment (build 1.7.0_72-b14)Java HotSpot(TM) Server VM (build 24.72-b04, mixed mode) Gelf clients that send logs all based on NxLog last version installed on windows 32 and 64 bit (mostly Windows 2008/2013 Servers)
Date: Fri, 20 Feb 2015 06:28:17 -0800 From: notifications@github.com To: graylog2-server@noreply.github.com CC: andrea.consadori@hotmail.it Subject: Re: [graylog2-server] ERROR: org.graylog2.inputs.codecs.GelfCodec - Could not parse JSON! (#984)
The garbage in the log messages looks very suspicious. Could you please elaborate a little bit on your Graylog setup? Which version of Graylog are you running, how did you install it (manually, OS packages, virtual appliance…), which Java version are you using, and which operating system is Graylog running on?
— Reply to this email directly or view it on GitHub.
andreaconsadori
commented
9 years ago i also see the gelf connection encrease continuously Total connections: 459 (7 active) Show details the GelF Client giving issues is a Windows 2012 Server R2 - Italian with nxlog configured
kroepke
commented
9 years ago The payload does not look like JSON to me at all. Are you sure that nxlog is configured correctly?
The input connections you are referring to implies you are using a GELF TCP input, it is normal that the overall number of connections is increasing if nxlog closes its client connection after sending a message.
andreaconsadori
commented
9 years ago in my config i've Module xm_gelf under extension and Module im_msvistalog in input in, then in output out
Module om_tcp Host graylogipserver Port 12201 OutputType GELF
how can i debug if only some server create issue with gelf?
andreaconsadori
commented
9 years ago the issue seems to be this
com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 18)): only regular white space (\r, \n, \t) is allowed between tokens at [Source:
andreaconsadori
commented
9 years ago if i use udp in gelf client and input the issue disappear
andreaconsadori
commented
9 years ago i make several test and i found that with the same nxlog config if i use gelf_udp all works perfectly, if i use gelf_tcp (changing only Module om_udp to om_tcp) i start to have the problem.
2015-03-31T14:30:14.366+02:00 WARN [ProcessBuffer] Unable to process event
MessageEvent{raw=null, message=null}, sequence 6104949
java.lang.IllegalStateException: Failed to decompress the GELF message payload
at org.graylog2.inputs.codecs.gelf.GELFMessage.getJSON(GELFMessage.java:146)
at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:109)
at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:120)
at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:79)
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:56)
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:33)
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:138)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.graylog2.plugin.Tools.decompressZlib(Tools.java:195)
at org.graylog2.inputs.codecs.gelf.GELFMessage.getJSON(GELFMessage.java:133)
... 10 moreit seems tobe this error ---> #428
bernd
commented
9 years ago GZIP compressed GELF over TCP is not possible due to the \0 byte delimter as described in #428.
I am closing this issue now.