influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.6k stars 5.57k forks source link

Cannot use both json_name_key and json_query in mqtt_consumer plugin #6990

Closed fredrikIOT closed 3 years ago

fredrikIOT commented 4 years ago

Relevant telegraf.conf:

[[inputs.mqtt_consumer]]
  servers = ["tcp://localhost:1883"]
  qos = 0
  connection_timeout = "30s"

  topics = [
    "test/#"
  ]
  tag_keys = ["tag"]

  persistent_session = false
  client_id = ""

  data_format = "json"
  json_name_key = "name"
  json_query = "values"

[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  database = "test_db"
  skip_database_creation = false

System info:

Telegraf version: 1.13.2 Environments: Linux mint, running a docker contariner with telegraf 1.13.2.

Steps to reproduce:

  1. Send this json on mqtt, with mqtt_consumer configurated like described above: { "name": "test_name", "timestamp": "2020-Feb-05 08:06:56.976", "values": { "temp_outside": 0.05285325646400452, "temp_indside": 0.007113146595656872 } }

  2. The measurement name is not the value of the json key "name".

Expected behavior:

Measurement should be named according to the name key in the json message.

Actual behavior:

The measurement name is "mqtt_consumer", not the value in the "name" key.

Additional info:

If I comment out json_query, I get a measurement called "testname" (like expected), but then i get "values" as a prefix for all the field_keys ("values_temp_outside" and "values_temp_indside").

danielnelson commented 4 years ago

Unfortunately the json_query is ran first, making it impossible to access items outside of the query. I think the json parser is reaching it's limits, and needs to be redesigned soon.

sjwang90 commented 3 years ago

The new JSON_v2 parser (json_v2) has just been released in Telegraf 1.19 rc-1. Please download and test it out with your JSON files and open any bugs or feature gaps you may encounter.