elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.18k stars 3.49k forks source link

Logstash - Elasticsearch Filter Plugin field referencing #9184

Open krasekhi opened 6 years ago

krasekhi commented 6 years ago

Hi,

I am a bit confused on reference some fields , in the example below windows.service.name works as [windows][service][name] but in the "fields =>" section event_data.LogonType does not work. Is this a limitation that you can only reference fields such as "host" and "@timestamp" but not fields like event_data.####? I've tried all of these combinations below without success -

"event_data.LogonType"
[event_data][LogonType]
[event_data.LogonType]
"[event_data][LogonType]"
"[event_data.LogonType]" 
[event_data]LogonType
event_data[LogonType] 
**Config -**
filter{
    if [datatype] == "metrics" and [windows][service][name] == "winlogbeat" {
        elasticsearch {
            hosts => ["localhost:9200"]
            index => "last_logon"
            query => "host:%{[host]}"
            fields => { 
                "host" => "last_logon_host"
                "@timestamp" => "last_logon_time"
                "[event_data][LogonType]" => "last_logon_type"
                }
        }
    }
}

Thanks!

omarix commented 6 years ago

Hi , Me too I faced a trouble with nested fields with elasticsearch-plugin filter , i think it's not supported or is a bug,

  elasticsearch {
    hosts => ["localhost"]
    user => ["admin"]
    password => ["admin"]
    index => "db_settings"
   # query => "normalization.source:\"auditbeat\" AND normalization.function:\"user_login\" AND normalization.action.old:\"logged-in\""    <= doesn't work
    query => "source:\"auditbeat\" AND function:\"user_login\" AND action_old:\"logged-in\""  # it's works
 # fields => { "[event][action]" => "[event][action]" } <= doesn't work
    fields => { "action_new" => "new_action" }
  }

output

{
         "event" => {
        "function" => "user_login",
          "action" => "logged-in",
          "source" => "auditbeat"
    },
          "host" => "rsiem-elk",
    "@timestamp" => 2018-04-21T01:46:55.693Z,
    "new_action" => "login",
       "message" => "source=auditbeat, function=user_login, action=logged-in",
      "@version" => "1"
}

the same question has been asked in elastic forum https://discuss.elastic.co/t/getting-nested-fields-elasticsearch-filter/118223

Regards.