search

logstash-plugins / logstash-output-influxdb

Apache License 2.0
58 stars 77 forks source link

coerce_values fail to take fields from variable #70

Open tanuj83 opened 7 years ago

tanuj83 commented 7 years ago

In logstash version 5.5, using plugin version 5.0.2 and after collecting data logstash has field name "metric_path" which is a data point in my influxdb

It is noticed that the variable "%{metric_path}" in the coerce_values is not working for plugin version 5.0.2

coerce_values => { "%{metric_path}" => "float" }

if I use direct path then it works for that metric

coerce_values => { "mm.GarbageCollector.ParNew.CollectionCount" => "float" }

But in version 3.0.2 the below was working coerce_values => { "%{metric_path}" => "float" }

OS Version - CentOS 6.7 Logstash Version - 5.5

InfluxDB output Config

output {

stdout { codec => rubydebug }

influxdb {

host => "host1" port => 8086 user => "influx_user" password => "XXXXXXX" db => "mydb1" retention_policy => "my_retention" measurement => "measurement1" send_as_tags => ["host"] coerce_values => { "%{metric_path}" => "float" } data_points => { "%{metric_path}" => "%{metric_value_number}" "host" => "%{host}" }

} }

On running Logstash with debug mode, you can see that "metric_value_number" is a number in stdout but when it goes to influxDB it becomes string

InfluxDB output - values=>{"mm.GarbageCollector.ParNew.CollectionCount"=>"7576.0" stdout ->

{ "metric_value_number" => 7576.0, "path" => "/opt/jmxdata/", "@timestamp" => 2017-09-01T04:46:21.812Z, "@version" => "1", "host" => "source1", "eventtype" => "jmxmetrics", "metric_path" => "mm.GarbageCollector.ParNew.CollectionCount", "type" => nil }

tanuj83 commented 7 years ago

is this bug? coerce_values not accepting variable for column name in Logstash 5.x

sslupsky commented 7 years ago

Agreed, this is a bug.

I am experiencing this problem as well. We recently upgraded logstash to v5 from v2. During the process the influx output plugin has been upgraded. We noticed right away that using the mutate filter option or the coerce values output option does not cast the value type. The float values are interpreted as strings by influx as you have described here when you use a variable for the field key.