grok overwrite not working for empty strings

jsvd commented 8 years ago

migrates from https://github.com/elastic/logstash/issues/2590

So when I have this input 

input="<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]: ASDF"

  grok {      
    match => ["message", "<%{NUMBER}>%{TIMESTAMP_ISO8601:syslogTimestamp} %{SYSLOGHOST} %{DATA:container_name}(?:\[%{POSINT}\])?:%{SPACE}%{GREEDYDATA:message}"]
    overwrite => [ "message" ]
  }

message is = ASDF

if I have this input="<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]: "
or this="<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]:"

The message is not  " " or "" like displayed in http://grokdebug.herokuapp.com/

jsvd commented 8 years ago

how to reproduce:

input {
  generator { count => 1 message => "<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]: ASDF"}
  generator { count => 1 message => "<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]: "}
  generator { count => 1 message => "<14>2015-02-11T17:49:29Z logspout dev_ziservice_1[1]:"}
}
filter {
  grok {
    match => ["message", "<%{NUMBER}>%{TIMESTAMP_ISO8601:syslogTimestamp} %{SYSLOGHOST} %{DATA:container_name}(?:\[%{POSINT}\])?:%{SPACE}?%{GREEDYDATA:message}"]
    overwrite => [ "message" ]
  }
}
output {
  stdout { codec => rubydebug }
}

Jispry commented 5 years ago

you need to use keep_empty_captures => true

see stackoverflow

logstash-plugins / logstash-filter-grok

grok overwrite not working for empty strings #77