search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.45k stars 1.07k forks source link

Parsing issues better notification #2717

Open JulioQc opened 8 years ago

JulioQc commented 8 years ago

Expected Behavior

Notification in WebUI.

Current Behavior

Only logged in system logs. Need to proactively check logs for the issue.

Possible Solution

Notification under overview

Steps to Reproduce (for bugs)

  1. Create CSV parsing (i.e Exchange SMTP logs)
  2. Wait for it to gather some logs
  3. Check server's log for ERROR: org.graylog2.plugin.inputs.Extractor - Could not apply converter [csv] of extractor [a8f941c0-4f7e-11e6-bded-005056ae6767].

    Context

So this is just an improvement suggestion for when CSV parsing fails like this: 

2016-08-19_12:40:16.71865 2016-08-19 08:40:16,717 ERROR: org.graylog2.plugin.inputs.Extractor - Could not apply converter [csv] of extractor [a8f941c0-4f7e-11e6-bded-005056ae6767].
2016-08-19_12:40:16.71977 java.lang.NullPointerException
2016-08-19_12:40:16.72395       at java.util.HashMap.putMapEntries(HashMap.java:500) ~[?:1.8.0_77]
2016-08-19_12:40:16.72397       at java.util.HashMap.<init>(HashMap.java:489) ~[?:1.8.0_77]
2016-08-19_12:40:16.72488       at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:250) [graylog.jar:?]
2016-08-19_12:40:16.72627       at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:228) [graylog.jar:?]
2016-08-19_12:40:16.72868       at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:62) [graylog.jar:?]
2016-08-19_12:40:16.73078       at org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100) [graylog.jar:?]
2016-08-19_12:40:16.73164       at org.graylog2.buffers.processors.ServerProcessBufferProcessor.handleMessage(ServerProcessBufferProcessor.java:56) [graylog.jar:?]
2016-08-19_12:40:16.73304       at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:82) [graylog.jar:?]
2016-08-19_12:40:16.73491       at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:61) [graylog.jar:?]
2016-08-19_12:40:16.73563       at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35) [graylog.jar:?]
2016-08-19_12:40:16.73849       at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:139) [graylog.jar:?]
2016-08-19_12:40:16.73854       at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
2016-08-19_12:40:16.73991       at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]

Would be nice to be notified to adjust or change extractor without having to "tail" the server logs to spot those issues (I guess Graylog could stream it's own logs and alert, as a workaround) Not much impact except losing those badly structured logs.

Your Environment

JulioQc commented 8 years ago

Cheers :)