influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.49k stars 5.55k forks source link

mqtt_consumer: "client_trace = true" doesn't work in telegraf 1.31.0 #15692

Closed douginoz closed 1 month ago

douginoz commented 1 month ago

Relevant telegraf.conf

[[inputs.mqtt_consumer]]
  servers = ["tcp://192.168.1.20:1883"]
  topics = ["Tempest_windrose/test"]
  username = "blah"
  password = "blech"
  qos = 0
  data_format = "json"
  client_trace = true

[agent]
  debug = true
  quiet = false
  logfile = "/logs/telegraf.log"

Logs from Telegraf

2024-07-31T09:08:19Z I! Loading config: /etc/telegraf/telegraf.conf
2024-07-31T09:08:19Z E! error loading config file /etc/telegraf/telegraf.conf: plugin inputs.mqtt_consumer: line 67: configuration specified the fields ["client_trace"], but they were not used. This is either a typo or this config option does not exist in this version.

System info

ubuntu 23.10

Docker

moa@sophie:~$ cat docker_configs/grafana_suite.yml version: '3.9' services: telegraf: container_name: telegraf image: my-telegraf:July restart: unless-stopped privileged: true user: telegraf:993 network_mode: host labels:

Steps to reproduce

  1. Add the line " client_trace = true" to the [[inputs.mqtt_consumer]] section of telegraf.conf
  2. Ensure the [Agent] section has debug = true
  3. Try starting telegraf. It fails to start and repeats the same error message over and over. ...

I

Expected behavior

additional and critically needed debug information on why absolutely nothing is happening or being reported/logged for mqtt_consumer.

Actual behavior

Telegraf won't start with this line in the config.

Additional info

telegraf@sophie:/$ telegraf --version Telegraf 1.31.0 (git: HEAD@fbfaba05)

The source for this version indicates that client_trace should work, but it doesn't.

Currently there's no way to know why mqtt_consumer is doing nothing. The debug logs show that it connects:

2024-07-31T09:08:47Z I! Starting Telegraf 1.31.0 brought to you by InfluxData the makers of InfluxDB
2024-07-31T09:08:47Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 60 outputs, 6 secret-stores
2024-07-31T09:08:47Z I! Loaded inputs: apcupsd cpu disk diskio dns_query docker file hddtemp http_response kernel mem mqtt_consumer net netstat ping processes sensors smart snmp_trap swap system
2024-07-31T09:08:47Z I! Loaded aggregators: 
2024-07-31T09:08:47Z I! Loaded processors: enum
2024-07-31T09:08:47Z I! Loaded secretstores: 
2024-07-31T09:08:47Z I! Loaded outputs: influxdb_v2
2024-07-31T09:08:47Z I! Tags enabled: host=sophie
2024-07-31T09:08:47Z W! Deprecated inputs: 0 and 2 options
2024-07-31T09:08:47Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"sophie", Flush Interval:10s
2024-07-31T09:08:47Z D! [agent] Initializing plugins
2024-07-31T09:08:47Z W! DeprecationWarning: Value "false" for option "ignore_protocol_stats" of plugin "inputs.net" deprecated since version 1.27.3 and will be removed in 1.36.0: use the 'inputs.nstat' plugin instead for protocol stats
2024-07-31T09:08:47Z D! [agent] Connecting outputs
2024-07-31T09:08:47Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2024-07-31T09:08:47Z D! [agent] Successfully connected to outputs.influxdb_v2
2024-07-31T09:08:47Z D! [agent] Starting service inputs
2024-07-31T09:08:47Z I! [inputs.snmp_trap] Listening on udp://:1162
2024-07-31T09:08:47Z I! [inputs.mqtt_consumer] Connected [tcp://192.168.1.20:1883]
2024-07-31T09:08:57Z D! [outputs.influxdb_v2]  Wrote batch of 347 metrics in 56.002669ms

I know it's connecting because if I change the IP address in the telegraf conf, it logs that it couldn't connect. So it's connecting.

When I issue mosquitto_sub -h 192.168.1.20 -p 1883 -t "Tempest_windrose/#" -u "blah" -P "blech" -v

from within the telegraf docker container, it lists:

telegraf@sophie:/$ mosquitto_sub -h 192.168.1.20 -p 1883 -t "Tempest_windrose/#" -u "blah" -P "blech" -v
Tempest_windrose/test {"wind_speed": "5.0", "wind_direction": "NNW"}

and when I issue mosquitto_pub -h 192.168.1.20 -p 1883 -t "Tempest_windrose/test" -u "blah" -P "blech" -m '{"wind_speed": 5.0, "wind_direction": 34}'

from another machine, I can see the message appear in the docker container. So we know that the docker container is allowing or has access to read and write to the mqtt server.

From everything I can find, publishing the message above should work. The data is in json format. The topic is correct. But there's absolutely nothing being logged in debug or on the console apart from the initial connection message.

Worse, this was all working fine 8 days ago. Telegraf hasn't changed, but the data being sent to the mqtt server may have changed, but I can't tell in what way!

powersj commented 1 month ago

This was fixed in #15670 after I accidently removed it while working on the mqtt output. You can use a nightly build, downgrade to v1.31, or wait for v1.32.3.