search

c1328 / ahoy-mosquitto-telegraf-influx-grafana

AHOY-DTU (https://github.com/grindylow/ahoy) delivers power metrics from inverter via MQTT to a MQTT (mosquitto) server. Telegraf subscribes that topic (“inverter/#”) and writes everything to InfluxDB. The Dashboard consumes data from InfluxDB.
12 stars 1 forks source link

help needed: telegraf not collecting all fields #2

Closed rdxmb closed 1 year ago

rdxmb commented 1 year ago

Hello, great projects!

I have the problem that my mosquitto-fields are not collected by telegraf. Especially the interesting fields like P_DC. Here are some tries:

root@raspberrypi:~# mosquitto_sub -h localhost -t ahoy/# -v | grep P_DC
ahoy/total/P_DC 1.2
ahoy/total/P_DC 1.2
ahoy/HM-600/ch1/P_DC 0.4
ahoy/HM-600/ch2/P_DC 0.8
ahoy/HM-600/ch0/P_DC 1.2
ahoy/total/P_DC 1.2
ahoy/total/P_DC 1.2
ahoy/total/P_DC 1.2
ahoy/total/P_DC 1.2
ahoy/total/P_DC 1.2
^C
root@raspberrypi:~# cat /root/telegraf-mqtt.conf 
[global_tags]
[agent]
  interval = "1s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"
  hostname = ""
  omit_hostname = false
#  logfile = "/var/log/telegraf/telegraf.log"
  debug = true

# Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    'ahoy/+/+/+'
  ]
  qos = 0
  connection_timeout = "30s"
  username = ""
  password = ""
  data_format = "value"
  data_type = "float"

root@raspberrypi:~# telegraf --config /root/telegraf-mqtt.conf --test
2023-08-04T15:45:23Z I! Loading config: /root/telegraf-mqtt.conf
2023-08-04T15:45:23Z I! Starting Telegraf 1.27.1
2023-08-04T15:45:23Z I! Available plugins: 237 inputs, 9 aggregators, 28 processors, 23 parsers, 59 outputs, 4 secret-stores
2023-08-04T15:45:23Z I! Loaded inputs: mqtt_consumer
2023-08-04T15:45:23Z I! Loaded aggregators: 
2023-08-04T15:45:23Z I! Loaded processors: 
2023-08-04T15:45:23Z I! Loaded secretstores: 
2023-08-04T15:45:23Z W! Outputs are not used in testing mode!
2023-08-04T15:45:23Z I! Tags enabled: host=raspberrypi
2023-08-04T15:45:23Z D! [agent] Initializing plugins
2023-08-04T15:45:23Z D! [agent] Starting service inputs
2023-08-04T15:45:23Z I! [inputs.mqtt_consumer] Connected [tcp://127.0.0.1:1883]
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch1/YieldDay value=0 1691163923849238561
2023-08-04T15:45:23Z D! [agent] Stopping service inputs
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch1/YieldTotal value=46.065 1691163923867318700
2023-08-04T15:45:23Z D! [inputs.mqtt_consumer] Disconnecting [tcp://127.0.0.1:1883]
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch2/YieldDay value=0 1691163923867739680
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch2/YieldTotal value=26.298 1691163923868047665
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch0/YieldDay value=2170 1691163923872500453
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch0/YieldTotal value=45.98 1691163923879886101
2023-08-04T15:45:23Z D! [inputs.mqtt_consumer] Disconnected [tcp://127.0.0.1:1883]
2023-08-04T15:45:23Z D! [agent] Input channel closed
2023-08-04T15:45:23Z D! [agent] Stopped Successfully
root@raspberrypi:~# cat /root/telegraf-mqtt.conf
[global_tags]
[agent]
  interval = "1s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"
  hostname = ""
  omit_hostname = false
#  logfile = "/var/log/telegraf/telegraf.log"
  debug = true

# Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    "ahoy/#"
  ]
  qos = 0
  connection_timeout = "30s"
  username = ""
  password = ""
  data_format = "value"
  data_type = "float"

root@raspberrypi:~# telegraf --config /root/telegraf-mqtt.conf --test
2023-08-04T15:47:43Z I! Loading config: /root/telegraf-mqtt.conf
2023-08-04T15:47:43Z I! Starting Telegraf 1.27.1
2023-08-04T15:47:43Z I! Available plugins: 237 inputs, 9 aggregators, 28 processors, 23 parsers, 59 outputs, 4 secret-stores
2023-08-04T15:47:43Z I! Loaded inputs: mqtt_consumer
2023-08-04T15:47:43Z I! Loaded aggregators: 
2023-08-04T15:47:43Z I! Loaded processors: 
2023-08-04T15:47:43Z I! Loaded secretstores: 
2023-08-04T15:47:43Z W! Outputs are not used in testing mode!
2023-08-04T15:47:43Z I! Tags enabled: host=raspberrypi
2023-08-04T15:47:43Z D! [agent] Initializing plugins
2023-08-04T15:47:43Z D! [agent] Starting service inputs
2023-08-04T15:47:43Z I! [inputs.mqtt_consumer] Connected [tcp://127.0.0.1:1883]
2023-08-04T15:47:43Z D! [agent] Stopping service inputs
2023-08-04T15:47:43Z D! [inputs.mqtt_consumer] Disconnecting [tcp://127.0.0.1:1883]
2023-08-04T15:47:43Z E! [inputs.mqtt_consumer] Error in plugin: strconv.ParseFloat: parsing "0.6.9": invalid syntax
2023-08-04T15:47:43Z D! [inputs.mqtt_consumer] Disconnected [tcp://127.0.0.1:1883]
2023-08-04T15:47:43Z D! [agent] Input channel closed
2023-08-04T15:47:43Z D! [agent] Stopped Successfully
2023-08-04T15:47:43Z E! [telegraf] Error running agent: input plugins recorded 1 errors
root@raspberrypi:~# cat /root/telegraf-mqtt.conf
[global_tags]
[agent]
  interval = "1s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"
  hostname = ""
  omit_hostname = false
#  logfile = "/var/log/telegraf/telegraf.log"
  debug = true

# Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    "ahoy/+/+/+"
  ]
  qos = 0
  connection_timeout = "30s"
  username = ""
  password = ""
  data_format = "value"
  data_type = "float"

root@raspberrypi:~# telegraf --config /root/telegraf-mqtt.conf --test
2023-08-04T15:49:18Z I! Loading config: /root/telegraf-mqtt.conf
2023-08-04T15:49:18Z I! Starting Telegraf 1.27.1
2023-08-04T15:49:18Z I! Available plugins: 237 inputs, 9 aggregators, 28 processors, 23 parsers, 59 outputs, 4 secret-stores
2023-08-04T15:49:18Z I! Loaded inputs: mqtt_consumer
2023-08-04T15:49:18Z I! Loaded aggregators: 
2023-08-04T15:49:18Z I! Loaded processors: 
2023-08-04T15:49:18Z I! Loaded secretstores: 
2023-08-04T15:49:18Z W! Outputs are not used in testing mode!
2023-08-04T15:49:18Z I! Tags enabled: host=raspberrypi
2023-08-04T15:49:18Z D! [agent] Initializing plugins
2023-08-04T15:49:18Z D! [agent] Starting service inputs
2023-08-04T15:49:18Z I! [inputs.mqtt_consumer] Connected [tcp://127.0.0.1:1883]
> mqtt_consumer,host=raspberrypi,topic=ahoy/HM-600/ch1/YieldDay value=0 1691164158777388745
2023-08-04T15:49:18Z D! [agent] Stopping service inputs
2023-08-04T15:49:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://127.0.0.1:1883]
2023-08-04T15:49:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://127.0.0.1:1883]
2023-08-04T15:49:18Z D! [agent] Input channel closed
2023-08-04T15:49:18Z D! [agent] Stopped Successfull

What's the trick here?

EDIT: crosslink: https://community.influxdata.com/t/inputs-mqtt-customer-only-collecting-some-fields/31036

rdxmb commented 1 year ago

closing this in place of https://github.com/lumapu/ahoy/issues/1067