search

emsearcy / fluent-plugin-gelf

Buffered fluentd output plugin to GELF (Graylog2)
Apache License 2.0
33 stars 57 forks source link

strip when checking for short message to ensure compatibility with graylog #41

Closed kmcgovern-apixio closed 4 years ago

kmcgovern-apixio commented 4 years ago

when checking short message the current check does a .empty? check. ruby considers a string empty only if it is "". if there is only whitespace it considers the string to be there.

java on the other hand will trim whitespace to ensure the string is empty. This can cause errors on the graylog side for the short_message field missing. 

2020-07-10T07:29:28.107Z ERROR [DecodingProcessor] Error processing message RawMessage{id=16622f79-c27f-11ea-9d76-02638c637b88, journalOffset=5966869, codec=gelf, payloadSize=260, timestamp=2020-07-10T07:29:28.039Z, remoteAddress=/x.x.x.x:xxxx}
java.lang.IllegalArgumentException: GELF message <16622f79-c27f-11ea-9d76-02638c637b88> (received from <10.1.32.206:40430>) has empty mandatory "short_message" field.
    at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:258) ~[graylog.jar:?]
    at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:140) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
    at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]

this in turn will cause the event to not be indexed. This PR fixes this by removing leading and trailing whitespace when doing the checking if a string is empty.

https://github.com/emsearcy/fluent-plugin-gelf/blob/master/lib/fluent/plugin/out_gelf.rb#L93-L107 https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/StringUtils.html#isBlank-java.lang.CharSequence- https://github.com/Graylog2/graylog2-server/blob/786c3414e13f2fc8b4f4d35d9e8d0c53910ac902/graylog2-server/src/main/java/org/graylog2/inputs/codecs/GelfCodec.java#L257