influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.47k stars 5.55k forks source link

Update json_v2 parser to get field values out of arrays #9481

Open ghost opened 3 years ago

ghost commented 3 years ago

Relevant telegraf.conf:

data_format = "json_v2"
    [[inputs.mqtt_consumer.json_v2]]
        [[inputs.mqtt_consumer.json_v2.object]]
            path = "@this"
            disable_prepend_keys = true
           # tags = ["data"]
        [inputs.mqtt_consumer.json_v2.object.fields]
            cnt  = "int"
            data = "int"
            format  = "int"

System info:

Telegraf Version 1.19.0, Linux Ubuntu 20.04, InfluxDB 1.8 and Grafana 7.5.4

Docker

version: '3.6' 
services: 
  telegraf: 
    image: telegraf:1.19.0-alpine 

Steps to reproduce:

Expected behavior:

Im testing the new json_v2 parser with InfluxDB and Grafana in a TIG-Stack configuration managed by docker. The old json parser gave me the array values as field values:

old json parser

Actual behavior:

The new json parser doesn't allow arrays to get trough. The solution discussed in this issue #9381 only gives me the array values as tag values:

[[inputs.mqtt_consumer]]
data_format = "json_v2"
    [[inputs.mqtt_consumer.json_v2]]
        [[inputs.mqtt_consumer.json_v2.object]]
            path = "@this"
            disable_prepend_keys = true
            tags = ["data"]
        [inputs.mqtt_consumer.json_v2.object.fields]
            cnt  = "int"
            format  = "int"

new json parser

The GJSON Path should be "data". Thanks for the work @sspaink

Additional info:

sjwang90 commented 3 years ago

@danberg13 can you share a sample of your json you're querying?

ghost commented 3 years ago

yeah of course:

{ 
   "cnt" : 23,
   "data": [ 
     3,
     7,
     10,
     23
], 
"format": 0
}

With the GJSON Path = "data" im getting just the last value.

sjwang90 commented 3 years ago

@danberg13 I'm getting this as my output (reading in the sample JSON from a file)

> file,data=3 cnt=23i,format=0i 1626389159000000000
> file,data=7 cnt=23i,format=0i 1626389159000000000
> file,data=10 cnt=23i,format=0i 1626389159000000000
> file,data=23 cnt=23i,format=0i 1626389159000000000