influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.59k stars 5.56k forks source link

processors.converter doesn't work with mqtt_consumer #4744

Closed Dees7 closed 6 years ago

Dees7 commented 6 years ago

Relevant telegraf.conf:

[global_tags]
[agent]
  interval = "60s"
  round_interval = true
  metric_batch_size = 500
  metric_buffer_limit = 1000
  collection_jitter = "0s"
  flush_interval = "90s"
  flush_jitter = "0s"
  precision = ""
  debug = true
  quiet = false
  logfile = "/var/log/telegraf/telegraf.log"
  hostname = ""
  omit_hostname = false
[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "arduino"
  skip_database_creation = false
  retention_policy = ""
  write_consistency = "any"
  timeout = "5s"
  username = "telegraf"
  password = "pass"
[[processors.converter]]
 [processors.converter.fields] 
  float = ["POWER*"] 
[[processors.printer]] 
[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false
[[inputs.disk]]
  mount_points = ["/","/var/wal","/boot"]
  ignore_fs = ["devtmpfs", "devfs"]
[[inputs.diskio]]
[[inputs.kernel]]
[[inputs.mem]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]
[[inputs.net]]
  interfaces = ["wlan0","eth0"]
[[inputs.procstat]]
  pattern = "rsyslogd|wpa_supplicant|bluetoothd|dhcpcd|influxd|grafana|sshd|mosquitto|SCREEN|bash|screen|telegraf"
  user = "pi|telegraf|grafana|influxdb|mosquitto"
[[inputs.mqtt_consumer]]
  servers = ["tcp://localhost:1883"]
  qos = 0
  connection_timeout = "30s"
  topics = ["tele/#"]
  persistent_session = true
  client_id = "telegraf"
  username = "user"
  password = "pass"
  name_override = "mqttc"
  data_format = "json"

System info:

Telegraf unknown (git: master 3268937c) mosquitto is an MQTT v3.1 broker.

Steps to reproduce:

I have mqtt-messages like there:

tele/sonoff/STATE {"Time":"2018-09-25T10:36:51","Uptime":"1T15:09:47","hostname":"nodemcu","Vcc":2.822,"POWER":"1","Wifi":{"AP":1,"SSId":"195","RSSI":60,"APMac":"C8:D3:A3:00:00:00"}}
tele/sonoff/SENSOR {"Time":"2018-09-25T10:36:51","hostname":"nodemcu","DS18B20":{"Temperature":41.8},"BMP280":{"Temperature":22.2,"Pressure":990.2},"TempUnit":"C"}

There is "POWER":"1" (not "POWER":1) and telegraf can't push it "as is" to influx. I tring convert with an "additionally config":

[[processors.converter]]
 [processors.converter.fields] 
  float = ["POWER*"] 

There is field POWER exist in the databese already and has type 'float' (from\for other clients) and I'd like to write mine values in that field.

Telegraf doesn't see metric POWER

Sep 25 10:36:51 piduino telegraf[10776]: mqttc,host=piduino,topic=tele/sonoff/STATE Wifi_RSSI=60,Vcc=2.822,Wifi_AP=1 1537853811162918278
Sep 25 10:36:51 piduino telegraf[10776]: mqttc,host=piduino,topic=tele/sonoff/SENSOR DS18B20_Temperature=41.8,BMP280_Temperature=22.2,BMP280_Pressure=990.2 1537853811216970795

Why? Is it possible to convert mqtt_consumer inputs?

danielnelson commented 6 years ago

Yes it should work with any input. I believe the problem here is that by default the JSON data format does not capture string fields. I believe you will want to use json_string_fields. Check this document for more info, but keep in mind some of these options require Telegraf 1.8.0.