logstash-plugins / logstash-filter-date

Apache License 2.0
7 stars 43 forks source link

Date format MMMdyyyy unable to be parsed #133

Closed mpg-13 closed 3 years ago

mpg-13 commented 4 years ago
input {
        stdin {}
}
filter {
        date {
                 match => ["message", "MMMdyyyy"]
        }
}
output {
        stdout {}
}

You can use any dates in the following formats to reproduce this issue Sep32019 and Sep132019

Using this config, logstash is able to properly parse out the date format Sep132019 :

{
    "@timestamp" => 2019-09-13T05:00:00.000Z,
       "message" => "Sep132019",
          "host" => "elkdevlogstash03",
      "@version" => "1"
}

but we get _dateparsefailure for Sep32019 :

{
    "@timestamp" => 2020-01-24T17:56:30.386Z,
       "message" => "Sep32019",
          "host" => "elkdevlogstash03",
      "@version" => "1",
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
TheVastyDeep commented 3 years ago

This is not an issue in the date filter, it is an issue in the underlying Joda library. The builder will consume 1 or 2 digits, regardless of whether you use d or dd. So it is trying to parse September 32nd, which is not a valid date. You may be able to use mutate to gsub in a delimiter between the day and year.

mutate { gsub => [ "message", "\d{4}$", ".\1" ] }