logstash-plugins / logstash-filter-mutate

Apache License 2.0
16 stars 75 forks source link

convert feature of mutate filter doesnot work while refering to the field values using % #30

Open jordansissel opened 9 years ago

jordansissel commented 9 years ago

(This issue was originally filed by @VarunMaheshwari at https://github.com/elastic/logstash/issues/2797)


example: "message" => "SolrTime,20"

my configuration looks like following:

filter { grok{ match => ["message","%{WORD:component},%{INT:time:int}"] } mutate{ add_field => [ "%{component}", "%{time}" ] } mutate{ convert => ["%{component}", "integer"] remove_field => ["component","time"] } }

my output should be something like: "SolrTime" => 20, ... But the output I am getting is "SolrTime" => "20", ...

that is type conversion didn't happen for %{component}, i.e. SolrTime field should have an integer value instead of String.

Is it a bug or is this feature not supported ?

purbon commented 9 years ago

@VarunMaheshwari I'm sorry it took us to much time to come back here, I was looking at this issue you posted here and what you can actually do is using something like:

filter {
grok{
match => ["message","%{WORD:component},%{INT:time:int}"]
}
mutate{
add_field => [ "[component]", "%{time}" ]
}
mutate{
convert => ["[component]", "integer"]
remove_field => ["component","time"]
}
}

then the conversion is going to work as you expected. More examples can be found in the test suite for example in https://github.com/logstash-plugins/logstash-filter-mutate/blob/master/spec/filters/mutate_spec.rb#L375-L447

let me know if you have more questions...

Faheemul commented 8 years ago

when we are converting some values to integer, if value is null then we see its value as 0, which gives wrong information in case where for some values we get values as 0.

EX: if message is SignalStrength= then we see #SignalStrength 0 as KV pair we expect blank vlaue.

since in some message we get SignalStrength=0 then we cannot differentiate between if the value is real 0 or it is created by logtsatsh as 0 when there is no data in value.