search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.31k stars 1.05k forks source link

GELF message has empty mandatory "short_message" field. #5793

Open mahnunchik opened 5 years ago

mahnunchik commented 5 years ago

Current Behavior

I've a lot of errors in log:

GELF message has empty mandatory "short_message" field.

Possible Solution

It is known problem of docker https://github.com/docker/for-linux/issues/354

Your Environment

chuegel commented 4 years ago

Possible related to https://github.com/Graylog2/graylog2-server/issues/6488

denis-zvonov commented 4 years ago

I am getting the same after migration to Graylog 3.1.

mboremski commented 4 years ago

I can confirm this Bug. But I have other circumstances. Using nxlog-ce to send GELF_TCP. I also wrote the GELF_TCP-Message to a file an got this:

{"version":"1.1",
"_EventReceivedTime":"2020-08-24 11:04:02",
"_SourceModuleName":"redmine_production",
"_SourceModuleType":"im_file",
"_SourceName":"Redmine Production",
"host":"VEP01",
"short_message":"Started GET \"**********" for 127.0.0.1 at 2020-08-24 11:04:01 +0200\r\nProcessing by TimeLoggersController#render_menu as */*\r\n  Parameters: {\"_\"=>\"1598252400620\"}\r\n  Current user: **** (id=**)\r\n  Rendered ******************** (11.3ms)\r\nCompleted 200 OK in 20ms (Views: 9.9ms | ActiveRecord: 5.7ms)\r",
"timestamp":1598259842,
"level":6}

As you can see the short_message is there. But graylog says:

2020-08-24T11:04:02.121+02:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=**********, journalOffset=260784, codec=gelf, payloadSize=233, timestamp=2020-08-24T09:04:02.120Z, remoteAddress=/192.168.1.***:61723} on input <5f40331381d87961ab292a56>.
2020-08-24T11:04:02.121+02:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=c0fcfc81-e5e8-11ea-aa55-00505681aa85, journalOffset=260784, codec=gelf, payloadSize=233, timestamp=2020-08-24T09:04:02.120Z, remoteAddress=/192.168.1.***:61723}
java.lang.IllegalArgumentException: GELF message <*****************> (received from <192.168.1.***:61723>) has empty mandatory "short_message" field.
    at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:258) ~[graylog.jar:?]
    at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:140) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?]
    at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]

Also noteable might be, the message is prcessed correctly by graylog.

Maybe this helps to solve this issue.

h4wkmoon commented 3 years ago

My graylog logs are filled with these message. It's about 10K lines each minute.

Maybe recreate the input is a workaround.

DilaraSSS commented 2 years ago

I have the same problem with my Graylog 4.0.11+e4e88a4, codename Noir. Any update?

Here are the logs :

2022-01-14 09:48:04,082 ERROR [DecodingProcessor] - Unable to decode raw message RawMessage{id=119053b1-751f-11ec-96f4-7af235b1e3ab, journalOffset=198692353, codec=gelf, payloadSize=514, timestamp=2022-01-14T09:48:04.075Z, remoteAddress=/10.40.0.0:19396} on input <607e93166eddf7582244c00f>. - {} 2022-01-14 09:48:04,082 ERROR [DecodingProcessor] - Error processing message RawMessage{id=119053b1-751f-11ec-96f4-7af235b1e3ab, journalOffset=198692353, codec=gelf, payloadSize=514, timestamp=2022-01-14T09:48:04.075Z, remoteAddress=/10.40.0.0:19396} - {} java.lang.IllegalArgumentException: GELF message <119053b1-751f-11ec-96f4-7af235b1e3ab> (received from <10.40.0.0:19396>) has empty mandatory "short_message" field. at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:258) ~[graylog.jar:?] at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:140) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:149) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_302]

storm1kk commented 2 years ago

The same on Graylog v4.2.6. Any updates on the subj?

elad-bar commented 2 years ago

same here with v4.3.3

Joshocan commented 1 year ago

Same here with v4.3.8

gespinal commented 2 weeks ago

I'm also receiving this on GrayLog v6.0