influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.63k stars 5.58k forks source link

Processors.Parser with parser "value" generates a new field called "value" #14346

Closed dmorigin closed 11 months ago

dmorigin commented 11 months ago

Please direct all support questsions to slack or the forums. Thank you.

Telegraf Version: 1.28.5 / Ubuntu Linux 22.04 (jammy)

I have the following configuration

[[processors.parser]]
    namepass = ["unbound_replies"]
    parse_fields = ["pid", "thread", "bytes"]
    drop_original = false
    merge = "override"
    data_format = "value"
    data_type = "integer"

In my opinion: The fields "pid", "thread" and "bytes" are gathered from the metric, parsed as an "integer" and then merged back to the original metric. But this isn't happend. Instead: One, or all fields are parsed as an integer and only one single new field is generated. Called "value", which stores (maybe?) the last parsed value (maybe my field: "bytes") and is attached to the original metric.

Now, the output (in my case InfluxDb v2) metric contains the original values (they are all of type "string") and the unwanted field "value" as integer.

In "plugins/parsers/value/parser.go" there is a line (38 - 40) which looks like this

    if v.FieldName == "" {
        v.FieldName = "value"
    }

It this the issue here?

At the moment I have to do the following workaround to fix this:

[[processors.parser]]
    namepass = ["unbound_replies"]
    parse_fields = ["pid"]
    drop_original = false
    merge = "override"
    data_format = "value"
    data_type = "integer"
    value_field_name = "pid"

[[processors.parser]]
    namepass = ["unbound_replies"]
    parse_fields = ["thread"]
    drop_original = false
    merge = "override"
    data_format = "value"
    data_type = "integer"
    value_field_name = "thread"

[[processors.parser]]
    namepass = ["unbound_replies"]
    parse_fields = ["bytes"]
    drop_original = false
    merge = "override"
    data_format = "value"
    data_type = "integer"
    value_field_name = "bytes"

What am I misunderstanding here?

telegraf-tiger[bot] commented 11 months ago

Hello! I recommend posting this question in our Community Slack or Community Forums, we have a lot of talented community members there who could help answer your question more quickly. You can also learn more about Telegraf by enrolling at InfluxDB University for free!

Heads up, this issue will be automatically closed after 7 days of inactivity. Thank you!

srebhan commented 11 months ago

@dmorigin any reason you are not using the converter processor? Like

# Convert values to another metric value type
[[processors.converter]]
  namepass = ["unbound_replies"]
  [processors.converter.fields]
    integer = ["pid", "thread", "bytes"]

The value parser works as expected and the only way around it is your workaround shown above... But I really think you should use the converter processor...

dmorigin commented 11 months ago

I saw the "converter" Option next day. Yes, it's an option instead of using the "value processor". I'm confused about, that it is possible to set multiple input field, but only one output field. Maybe you can explain what's the intention behind this is?

srebhan commented 11 months ago

Well for the value parser it makes sense to only have one output field, it does not know that you use it in the parser processor... :-)