search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.4k stars 1.06k forks source link

Graylog Error on inport content pack with Syslog UDP Input #10151

Open joaopalma5 opened 3 years ago

joaopalma5 commented 3 years ago

I create a content pack using docker with image: graylog/graylog:4.0 and now I'm importing into image: graylog/graylog:4.0 and image: graylog/graylog:4.0.5 but always have the same error: graylog_1 | 2021-02-24 22:50:52,749 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <6036d84ad48fe83f83cb8638> misfired. Reason: null

Expected Behavior

Import without errors

Current Behavior

Part of content pack with syslog udp input: { "v": "1", "type": { "name": "input", "version": "1" }, "id": "6619df96-f068-4257-b002-1952c08f2d78", "data": { "title": { "@type": "string", "@value": "Nginx" }, "configuration": { "port": { "@type": "integer", "@value": 12206 }, "recv_buffer_size": { "@type": "integer", "@value": 262144 }, "force_rdns": { "@type": "boolean", "@value": false }, "allow_override_date": { "@type": "boolean", "@value": true }, "override_source": { "@type": "string", "@value": "Nginx" }, "bind_address": { "@type": "string", "@value": "0.0.0.0" }, "expand_structured_data": { "@type": "boolean", "@value": false }, "store_full_message": { "@type": "boolean", "@value": false }, "number_worker_threads": { "@type": "integer", "@value": 4 } }, "static_fields": { "from_nginx": { "@type": "string", "@value": "true" } }, "type": { "@type": "string", "@value": "org.graylog2.inputs.syslog.udp.SyslogUDPInput" }, "global": { "@type": "boolean", "@value": false }, "extractors": [ { "target_field": { "@type": "string", "@value": "" }, "condition_value": { "@type": "string", "@value": "" }, "order": { "@type": "integer", "@value": 0 }, "converters": [], "configuration": { "grok_pattern": { "@type": "string", "@value": "%{DATA} %{DATA} %{GREEDYDATA:json}" }, "named_captures_only": { "@type": "boolean", "@value": true } }, "source_field": { "@type": "string", "@value": "message" }, "title": { "@type": "string", "@value": "JSON Extract from message" }, "type": { "@type": "string", "@value": "GROK" }, "cursor_strategy": { "@type": "string", "@value": "COPY" }, "condition_type": { "@type": "string", "@value": "NONE" } }, { "target_field": { "@type": "string", "@value": "" }, "condition_value": { "@type": "string", "@value": "^\\{" }, "order": { "@type": "integer", "@value": 0 }, "converters": [], "configuration": { "flatten": { "@type": "boolean", "@value": true }, "list_separator": { "@type": "string", "@value": ", " }, "kv_separator": { "@type": "string", "@value": "=" }, "key_prefix": { "@type": "string", "@value": "" }, "key_separator": { "@type": "string", "@value": "_" }, "replace_key_whitespace": { "@type": "boolean", "@value": false }, "key_whitespace_replacement": { "@type": "string", "@value": "_" } }, "source_field": { "@type": "string", "@value": "json" }, "title": { "@type": "string", "@value": "Extract JSON fields" }, "type": { "@type": "string", "@value": "JSON" }, "cursor_strategy": { "@type": "string", "@value": "CUT" }, "condition_type": { "@type": "string", "@value": "REGEX" } } ] }, "constraints": [ { "type": "server-version", "version": ">=4.0.2+1987d10" } ] },

LOGS:

graylog_1  | 2021-02-24 22:50:52,472 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/6036d84ad48fe83f83cb8615] is now RUNNING
graylog_1  | 2021-02-24 22:50:52,702 WARN : org.graylog2.inputs.transports.UdpTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=Storage, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=0a98c7b5-c103-41d3-a70b-0f2aeae919c7} (channel [id: 0xc6cf540b, L:/0.0.0.0:12205]) should be 262144 but is 425984.
graylog_1  | 2021-02-24 22:50:52,717 WARN : org.graylog2.inputs.transports.UdpTransport - receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=Nginx, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=0a98c7b5-c103-41d3-a70b-0f2aeae919c7} (channel [id: 0xf5ade70b, L:/0.0.0.0:12206]) should be 262144 but is 425984.
graylog_1  | 2021-02-24 22:50:52,764 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/6036d84ad48fe83f83cb861d] is now RUNNING

graylog_1  | 2021-02-24 22:50:52,749 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <6036d84ad48fe83f83cb8638> misfired. Reason: null
graylog_1  | org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
graylog_1  |    at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
graylog_1  |    at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
graylog_1  |    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
graylog_1  |    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_282]
graylog_1  |    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_282]
graylog_1  |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_282]
graylog_1  |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_282]
graylog_1  |    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
graylog_1  | Caused by: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
graylog_1  |    at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:135) ~[graylog.jar:?]
graylog_1  |    at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
graylog_1  |    ... 7 more
graylog_1  | Caused by: java.lang.NullPointerException
graylog_1  |    at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:130) ~[graylog.jar:?]
graylog_1  |    at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
graylog_1  |    ... 7 more
graylog_1  | 2021-02-24 22:50:52,998 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/6036d84ad48fe83f83cb8638] is now FAILED

graylog_1  | 2021-02-24 22:50:53,000 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/6036d84ad48fe83f83cb8643] is now STARTING

One note: Exist one WARN message in all inputs: WARN : org.graylog2.inputs.transports.UdpTransport - receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=Nginx, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=0a98c7b5-c103-41d3-a70b-0f2aeae919c7} (channel [id: 0xf5ade70b, L:/0.0.0.0:12206]) should be 262144 but is 425984. But as you can see in json of contentpack the recv_buffer_size is set to 262144 "recv_buffer_size": { "@type": "integer", "@value": 262144 },

Steps to Reproduce (for bugs)

  1. With docker image image: graylog/graylog:4.0 create a content pack with one Syslog UDP input
  2. Import in other installation with same image
  3. In my case if I go to System/Inputs -> Inputs and see the syslog UDP input state is NOT RUNNING
  4. Go to docker logs and see if the error is present in logs..
joaopalma5 commented 3 years ago

With my experiences, I find one problem in extractors, look in content_pack.json has null values... "extractors": [ { "target_field": { "@type": "string", "@value": "" }, "condition_value": { "@type": "string", "@value": "" }, "order": { "@type": "integer", "@value": 0 },

I have two extractor one grok pattern and another json... I try manual delete the two extractors from json and import but the error still the same... :/