search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.43k stars 1.07k forks source link

NullPointerException parsing GELF messages #5174

Closed Andrew-Stubbs closed 4 years ago

Andrew-Stubbs commented 6 years ago

I have setup a new UDP GELF input on my graylog server (Centos7 running docker image graylog/graylog:2.4.5-1), but cannot get any messages to come through.

I have tried integrating with an upstream system, and also tried using netcat to send messages in, as described in the graylog documentation:

echo -n '{ "version": "1.1", "host": "example.org", "short_message": "A short message", "level": 5, "_some_info": "foo" }' | nc -w0 -u my.graylog.example.com 12201

But through either input, I see nothing in the UI, and see the following in the logs:

2018-10-04 10:29:27,500 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Unable to decode raw message RawMessage{id=5eec41b0-c7c0-11e8-a7c5-0242ac110005, journalOffset=735640, codec=CEF, payloadSize=112, timestamp=2018-10-04T10:29:27.499Z, remoteAddress=/172.26.168.178:33118} on input <5bb5c780cff47e00014ad4b7>. 2018-10-04 10:29:27,500 ERROR: org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing message RawMessage{id=5eec41b0-c7c0-11e8-a7c5-0242ac110005, journalOffset=735640, codec=CEF, payloadSize=112, timestamp=2018-10-04T10:29:27.499Z, remoteAddress=/172.26.168.178:33118} java.lang.NullPointerException: null at org.graylog.plugins.cef.parser.MappedMessage.(MappedMessage.java:21) ~[?:?] at org.graylog.plugins.cef.codec.CEFCodec.decodeCEF(CEFCodec.java:108) ~[?:?] at org.graylog.plugins.cef.codec.CEFCodec.decode(CEFCodec.java:97) ~[?:?] at > org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_72-internal]

Is there something I've missed to be able to receive GELF over UDP?

danotorrey commented 5 years ago

Note that the source for the CEFCodec.java is located in the plugin repo https://github.com/Graylog2/graylog-plugin-cef/ in Graylog 2.4.x.

dennisoelkers commented 4 years ago

@Andrew-Stubbs: Is this still an issue for you?

hatifnatt commented 4 years ago

I can confirm this issue. First our developers tried to send messages using graylog2/gelf-php after their attempts failed I tried to send message via netcat and this attempt failed too.

GELF input configured with newline delimiter. Some system info.

Debian 10
Graylog 3.3.2+ade4779
openjdk-11-jre-headless:amd64 11.0.7+10-3~deb10u1

netcat example

echo -e '{ "version": "1.1", "host": "example.org", "short_message": "A short message", "level": 5, "_some_info": "foo" }' | nc -c -w1 my.graylog.server 5555

Error for message above.

2020-07-14T17:01:41.184+03:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=8ae1fcf0-c5da-11ea-8f3f-4edb742db22f, journalOffset=47034, codec=CEF, payloadSize=112, timestamp=2020-07-14T14:01:41.183Z, remoteAddress=/10.17.1.253:43550} on input <5f036afaafaf1e09a0adf0f0>.
2020-07-14T17:01:41.184+03:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=8ae1fcf0-c5da-11ea-8f3f-4edb742db22f, journalOffset=47034, codec=CEF, payloadSize=112, timestamp=2020-07-14T14:01:41.183Z, remoteAddress=/10.17.1.253:43550}
java.lang.NullPointerException: null
        at org.graylog.plugins.cef.parser.MappedMessage.<init>(MappedMessage.java:37) ~[graylog.jar:?]
        at org.graylog.plugins.cef.codec.CEFCodec.decodeCEF(CEFCodec.java:128) ~[graylog.jar:?]
        at org.graylog.plugins.cef.codec.CEFCodec.decode(CEFCodec.java:117) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
        at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]

Also tried to use input with Null frame delimiter? enabled, but error is similar.

echo -n -e '{ "version": "1.1", "host": "example.org", "short_message": "A short message", "level": 5, "_some_info": "foo" }'"\0" | nc -c -w1 my.graylog.server 5555
2020-07-14T17:09:58.146+03:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=b3185f10-c5db-11ea-8f3f-4edb742db22f, journalOffset=47061, codec=CEF, payloadSize=112, timestamp=2020-07-14T14:09:58.145Z, remoteAddress=/10.17.1.253:43782} on input <5f036afaafaf1e09a0adf0f0>.
2020-07-14T17:09:58.146+03:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=b3185f10-c5db-11ea-8f3f-4edb742db22f, journalOffset=47061, codec=CEF, payloadSize=112, timestamp=2020-07-14T14:09:58.145Z, remoteAddress=/10.17.1.253:43782}
java.lang.NullPointerException: null
        at org.graylog.plugins.cef.parser.MappedMessage.<init>(MappedMessage.java:37) ~[graylog.jar:?]
        at org.graylog.plugins.cef.codec.CEFCodec.decodeCEF(CEFCodec.java:128) ~[graylog.jar:?]
        at org.graylog.plugins.cef.codec.CEFCodec.decode(CEFCodec.java:117) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
        at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]
hatifnatt commented 4 years ago

I also have old Graylog 3.0 server used for testing some time ago. It successfully parses same messages. System info

Debian 9.12
Graylog 3.0.2+1686930
Oracle Java 1.8.0_191-b12

Main difference is Graylog version and Java version.

Probably related #8370

bernd commented 4 years ago

@hatifnatt Can you please show us the input configuration for the input that is listening on port 5555?

hatifnatt commented 4 years ago

@bernd of course

bind_address: 0.0.0.0
locale: <empty>
max_message_size: 2097152
number_worker_threads: 2
port: 5555
recv_buffer_size: 1048576
tcp_keepalive: false
timezone: Europe/Moscow
tls_cert_file: <empty>
tls_client_auth: disabled
tls_client_auth_cert_file: <empty>
tls_enable: false
tls_key_file: <empty>
tls_key_password: ********
use_full_names: false
use_null_delimiter: false

most settings are on default values.

bernd commented 4 years ago

@hatifnatt Thank you! What is the input type?

hatifnatt commented 4 years ago

@bernd I feel pretty dumb now... All that time I stared to input type and it was CEF TCP Input and that fact didn’t bother me for unknown reason. Yesterday I ran Graylog in Docker and found that GELF working fine, but I thought that was due different Java version. I just created new input with correct type and it working without any problems. I'm very sorry for for the inconvenience. I hope this issue will help other people with similar "problem" in future :)

bernd commented 4 years ago

@hatifnatt No worries, I am glad it works for you now. :slightly_smiling_face: