logstash-plugins / logstash-filter-date

Apache License 2.0
7 stars 43 forks source link

date filter fails to parse "/" when using locale #112

Open mostolog opened 7 years ago

mostolog commented 7 years ago

As described below, date filter doesn't seem to properly implement http://www.joda.org/joda-time/key_format.html when using locale.

https://github.com/elastic/logstash/issues/8491

mostolog commented 7 years ago

Further info:

logstash -e "input { stdin { type => stdin } } filter { mutate { add_field => { \"date\" => \"16/Oct/2017:16:27:26 +0200\" } } date { timezone => \"Europe/Madrid\" match => [\"date\", \"dd'/'MMM'/'yyyy':'HH':'mm':'ss' 'Z\"] locale => \"es\" tag_on_failure => \"error_date\" } } output { stdout { codec => rubydebug } }"

shows:

Sending Logstash's logs to /logstash/logs which is now configured via log4j2.properties
[2017-10-16T14:37:49,972][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/logstash/modules/fb_apache/configuration"}
[2017-10-16T14:37:49,979][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/logstash/modules/netflow/configuration"}
[2017-10-16T14:37:50,000][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/logstash/data/queue"}
[2017-10-16T14:37:50,002][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/logstash/data/dead_letter_queue"}
[2017-10-16T14:37:50,059][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"07f88ca8-907a-4d9e-a1cb-7fb1c199966d", :path=>"/logstash/data/uuid"}
[2017-10-16T14:37:53,046][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-10-16T14:37:53,292][INFO ][logstash.pipeline        ] Pipeline main started
The stdin plugin is now waiting for input:
[2017-10-16T14:37:53,479][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Writing "a" on input results in:

{
          "date" => "16/Oct/2017:16:27:26 +0200",
    "@timestamp" => 2017-10-16T14:38:06.448Z,
      "@version" => "1",
          "host" => "7d0847e692f9",
       "message" => "a",
          "type" => "stdin",
          "tags" => [
        [0] "error_date"
    ]
}