logstash-plugins / logstash-filter-csv

Apache License 2.0
15 stars 41 forks source link

logstash-7.1.0 stores headers as value #83

Open giovannicadiz opened 4 years ago

giovannicadiz commented 4 years ago

I have a problem loading data with losgatsh this when executing logshtash.bat only stores headers as values. Previously it had 2 pipes and now only 1 but it is not solved. I have tried several filters but it does not work

  1. Input csv:

ipaddress;name;isp;state;isreachable;operativo;request;response;total_seconds;enviados;recibidos;perdidos;performance 127.0.0.1;TD-26;Nextel;Initialized;1;UP;2020-05-15 01:30:02.278598;2020-05-15 01:30:07.387343;5.108745;5;5;0;100.0

  1. config file:

input { file { start_position => beginning path => "C:/ELK/logstash-7.1.0/data/td/ping/*.csv" sincedb_path => "C:/ELK/logstash-7.1.0/sincedb/sincedb_app_concentrator_ping/sincedb.log" } } filter { csv { separator => ';'

autodetect_column_names => true

    columns => ["ipaddress", "name", "isp", "state", "isreachable", "operativo", "request", "response", "total_seconds", "enviados", "recibidos", "perdidos", "performance"]
}

# mutate {
#   remove_field => ["message", "@version", "_id", "_score", "_type", "host", "path"]
# }

} output {

elasticsearch {
    hosts => "localhost:9200"
    index => "index_concentrator_ping"
    user => logstash_internal 
    password => logstash_internal

}
stdout {codec => rubydebug}

}

  1. stdout:

{ "isreachable" => "isreachable", "operativo" => "operativo", "name" => "name", "isp" => "isp", "perdidos" => "perdidos", "message" => "ipaddress;name;isp;state;isreachable;operativo;request;response;total_seconds;enviados;recibidos;perdidos;performance\r", "@version" => "1", "path" => "C:/ELK/logstash-7.1.0/data/td/ping/temp_TD-105941.csv", "host" => "CHLEDX10S028430", "total_seconds" => "total_seconds", "state" => "state", "enviados" => "enviados", "request" => "request", "@timestamp" => 2020-05-15T17:11:56.832Z, "recibidos" => "recibidos", "ipaddress" => "ipaddress", "response" => "response", "performance" => "performance" }

any solution?