feat (processors.execd): introduce buffer_size config option

SirBreadc commented 1 month ago

Relevant telegraf.conf

[[inputs.netflow]]
  metric_buffer_limit = 500000
  flush_interval = "60s"
  service_address = "udp4://:2055"
  protocol = "ipfix"
  private_enterprise_number_files = ["telegraf_conf/netflow_ingestion/custom_fields.csv"]
  dump_packets = false

[[processors.execd]]
  command = ["telegraf_conf/netflow_ingestion/netflow_options_packet_parser.py"]
  namepass = ["netflow_options"]
  buffer_size ="1Mib"

Logs from Telegraf

2024-07-29T23:56:51Z E! error loading config file telegraf_conf/netflow_ingestion/telegraf_netflow copy.conf: plugin processors.execd: line 70: configuration specified the fields ["buffer_size"], but they were not used. This is either a typo or this config option does not exist in this version.

System info

1.32.0

Docker

No response

Steps to reproduce

add buffer_size ="1Mib" to your telegarf file and run it
...

Expected behavior

We'd like a similar option to what is seen here https://github.com/influxdata/telegraf/blob/release-1.31/plugins/inputs/execd/README.md the reason for this ask is due to stderr messaged we need to log out are quite large, this is to assist us with debugging

Actual behavior

2024-07-29T23:56:51Z E! error loading config file telegraf_conf/netflow_ingestion/telegraf_netflow copy.conf: plugin processors.execd: line 70: configuration specified the fields ["buffer_size"], but they were not used. This is either a typo or this config option does not exist in this version.

Additional info

We are wanting to increase the stderr buffer to show more verbose logs atm we are seeing 2024-07-29T23:58:50Z E! [processors.execd] Error reading stderr: bufio.Scanner: token too long

powersj commented 1 month ago

buffer_size for execd processor not working

That's because it doesn't exist ;) we would call this a "feature request" not a bug. I'll take a look this week

SirBreadc commented 1 month ago

I thought about making this a feature request but I wasn't sure haha :) All good thanks for looking into it, let me know when it's ready for testing :)

powersj commented 1 month ago

Hey @SirBreadc - take a look at https://github.com/influxdata/telegraf/pull/15690 and give the artifacts a try in 20-30mins and let me know if it resolves the request.

SirBreadc commented 1 month ago

Hey @powersj sorry for the delay will check this out later today :) and report back if it worked as excepted

influxdata / telegraf