logstash-plugins / logstash-filter-grok

Grok plugin to parse unstructured (log) data into something structured.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Apache License 2.0
122 stars 97 forks source link

Using event fileds in configuration options #161

Open icetimidus opened 4 years ago

icetimidus commented 4 years ago

May I using logstash evnet fileds in grok filter configuration options?

Such as,

    grok { 
        patterns_dir => ["/opt/logstash/patterns/%{[@metadata][some_filed]}"]
        match => {"message" => "%{SOME_PATTERN}"}
    } 

or

    grok { 
        patterns_dir => ["/opt/logstash/patterns/sub"]
        patterns_files_glob => "%{[@metadata][some_filed]}"
        match => {"message" => "%{SOME_PATTERN}"}
    } 

I have try it in logstash 7.6.1 but faild. Any idea?

kares commented 3 years ago

this isn't supported atm, and I am not sure it ever will be. the current work-flow is load the pattern definitions once (for performance reasons) before any events are matched.

the closest one could do is having separate grok plugin instances if-ed based on event data e.g.

filter {
  if [some_field] {
    grok {  
      pattern_definitions => {
        "SOME_PATTERN" => '...'
      }
    }
  }
}