search

logstash-plugins / logstash-filter-grok

Grok plugin to parse unstructured (log) data into something structured.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Apache License 2.0
122 stars 97 forks source link

Logstash grok with empty match is impossible to debug #123

Open kub1x opened 6 years ago

kub1x commented 6 years ago

Hey, our Logstash pipeline config files are generated. There was a bug in our generation system which led to following grog pattern. The pattern was one of MANY patterns in our config and it contained override and other clauses as well, so it was not easy to notice the empty match pattern. 

filter {
  grok {
    match => [ "", "" ]
  }
}

On older versions (2.4 or so) it probably worked, or at least did not vail. After update, on Logstash v5.6.3 it throws following exception. The exception doesn't say what is wrong or where in the config file we did the mistake. 

LogStash::Instrument::MetricNoKeyProvided: LogStash::Instrument::MetricNoKeyProvided
     validate_key! at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/instrument/metric.rb:74
             gauge at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/instrument/metric.rb:31
             gauge at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/instrument/namespaced_metric.rb:32
          register at /home/kub1x/test/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.4.3/lib/logstash/filters/grok.rb:277
              each at org/jruby/RubyHash.java:1342
          register at /home/kub1x/test/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.4.3/lib/logstash/filters/grok.rb:275
          register at /home/kub1x/test/logstash-5.6.3/vendor/jruby/lib/ruby/1.9/forwardable.rb:201
   register_plugin at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/pipeline.rb:290
  register_plugins at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/pipeline.rb:301
              each at org/jruby/RubyArray.java:1613
  register_plugins at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/pipeline.rb:301
     start_workers at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/pipeline.rb:311
               run at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/pipeline.rb:235
    start_pipeline at /home/kub1x/test/logstash-5.6.3/logstash-core/lib/logstash/agent.rb:398

This is a bug in Logstash (I gusess in grok filter itself) as it should write a valuable message with a line number in config file instead of just throwing an exception.