Grok filter would benefit from type hinting in captures

[GoogleCodeExporter]

numerical values would benefit from some kind of type hinting. Hints like 'int' 
or 'float' will help make it so we can do better range queries in elasticsearch 
and allow other tools in the logstash pipeline to better handle number values.

on IRC we discussed options, and it was proposed that we abuse the grok pattern 
naming syntax %{PATTERN:name} and use %{PATTERN:name:type} where 'type' is 
'int' or 'float'

Original issue reported on code.google.com by jls.semi...@gmail.com on 24 Feb 2011 at 6:45

[GoogleCodeExporter]

Code for this is implemented.

Next task is to go through the grok patterns and add type hints to all the 
values that it makes sense to do so on.

Original comment by jls.semi...@gmail.com on 24 Feb 2011 at 8:10

[GoogleCodeExporter]

Also needs tests.

Original comment by jls.semi...@gmail.com on 2 Mar 2011 at 1:19

[GoogleCodeExporter]

tests written.

Original comment by jls.semi...@gmail.com on 2 Mar 2011 at 1:31

[GoogleCodeExporter]

i tried this, when my pattern defined as:
input {
  file {
    type => "tyler-syslog"
    path => [ "/var/log/messages*"]
  }
}

filter {
  grok {
    type => "tyler-syslog"
    pattern => "\w{3} \d{2} \d{2}:%{MINUTE:min:int}:"
  }
}
output {
  stdout {
    debug => true
  }
}

And my /var/log/messages contains content as:
Mar 28 01:20:05 MyVM ntpd[2262]: time reset -15.714794 s

However the result comes out as:
{"@source"=>"file://MyVM/var/log/messages", "@tags"=>[], 
"@fields"=>{"min"=>[20]}, "@timestamp"=>"2013-03-28T01:24:48.314Z", 
"@source_host"=>"MyVM", "@source_path"=>"/var/log/messages", "@message"=>"Mar 
28 01:20:05 MyVM ntpd[2262]: time reset -15.714794 s", "@type"=>"tyler-syslog"}

Referring to the codes: grok.rb, line 362-371:
            if !value.nil? && (!value.empty? rescue true)
            # Store fields as an array unless otherwise instructed with the
            # 'singles' config option
            if !event.fields.include?(key) and @singles
              event.fields[key] = value
            else
              event.fields[key] ||= []
              event.fields[key] << value
            end
          end

Please note that, the output is: "min"=>[20], actually, i thought it should be 
"min"=>["20",20], right? Please correct me if i miss anything.

Original comment by laitaiz...@gmail.com on 28 Mar 2013 at 3:19