Plugin doesn't add fields

[iShark5060]

Hi, I am using your plugin to add GeoIP data to my OPNsense telegraf outputs. My setup is as follows:

OPNsense telegraf plugin [influxDB2 output] -> telegraf (input.influxDB_v2_listener) -> processors.execd -> output.influxdb2_v2

Telegraf config:

[global_tags]

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "30s"
  flush_jitter = "0s"
  precision = ""
  hostname = ""
  omit_hostname = false

# Output to InfluxDB
[[outputs.influxdb_v2]]
  urls = ["http://10.50.60.51:8086"]
  token = "<omitted>"
  organization = "HomeLAN"
  bucket = "HomeLAN"

# GeoIP processing
[[processors.execd]]
  command = ["/etc/telegraf/geoip/geoip", "--config", "/etc/telegraf/geoip/GeoIP.conf"]

[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]

[[inputs.kernel]]

[[inputs.mem]]

[[inputs.processes]]

[[inputs.swap]]

[[inputs.system]]

# IPMI Input
[[inputs.ipmi_sensor]]
  servers = ["root:<omitted>@lanplus(10.50.60.150)","root:<omitted>@lanplus(10.50.60.151)"]
  interval = "120s"
  timeout = "60s"
  metric_version = 2

# OPNsense Input
[[inputs.influxdb_v2_listener]]
  service_address = ":8087"
  bucket_tag = "HomeLAN"
  token = "<omitted>"

# TrueNAS Input
[[inputs.socket_listener]]
  service_address = "tcp://:2003"
  data_format = "graphite"

/etc/telegraf/geoip/GeoIP.conf:

[[processors.geoip]]
## db_path is the location of the MaxMind GeoIP2 City database
db_path = "/usr/share/GeoIP/GeoLite2-City.mmdb"

[[processors.geoip.lookup]]
# get the ip from the field "source_ip" and put the lookup results in the respective destination fields (if specified)
field = "src_ip"
dest_country = "src_country"
dest_city = "src_city"
dest_lat = "src_lat"
dest_lon = "src_lon"

journalctl -u telegraf:

Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Starting Telegraf 1.21.4+ds1-0ubuntu2
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Loaded inputs: cpu disk influxdb_v2_listener ipmi_sensor kernel mem processes socket_listener swap system
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Loaded aggregators:
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Loaded processors: execd
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Loaded outputs: influxdb_v2
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! Tags enabled: host=monitoring
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"monitoring", Flush Interval:30s
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! [processors.execd] Starting process: /etc/telegraf/geoip/geoip [--config /etc/telegraf/geoip/GeoIP.conf]
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! [inputs.influxdb_v2_listener] Started HTTP listener service on :8087
Feb 25 21:38:36 monitoring telegraf[8286]: 2024-02-25T20:38:36Z I! [inputs.socket_listener] Listening on tcp://[::]:2003

(this had been running since around 2 hours - no errors)

Data is being pushed into the InfluxDB - but the fields are not added.

I can also see that the plugin is called kinda constantly...

Am I doing something terribly wrong? Thank you.

[iShark5060]

okay, I've added some Logging into the sourcecode and tracked around a bit. Seems like it doesn't find the fields (yes I know ... kinda crude):

Log output:

Feb 26 01:43:43 monitoring telegraf[9652]: 2024-02-26T00:43:43Z E! [processors.execd::SourceIP] stderr: "2024/02/26 01:43:43 E! Found nothing apparently: src_ip"
Feb 26 01:43:43 monitoring telegraf[9652]: 2024-02-26T00:43:43Z E! [processors.execd::SourceIP] stderr: "2024/02/26 01:43:43 E! This was empty somhow: src_ip"
Feb 26 01:43:43 monitoring telegraf[9652]: 2024-02-26T00:43:43Z E! [processors.execd::SourceIP] stderr: "2024/02/26 01:43:43 E! were done now. Data: [suricata map[dest_ip:10.50.60.1 dest_port:53 event_type:alert host:ChaosRouter path:/var/log/suricata/eve.json src_ip:10.50.60.100 src_port:65190] map[alert_action:allowed alert_category:Misc activity alert_gid:1 alert_metadata_created_at_0:2022_03_15 alert_metadata_former_category_0:INFO alert_metadata_signature_severity_0:Informational alert_metadata_updated_at_0:2022_11_30 alert_rev:4 alert_severity:3 alert_signature:ET INFO Observed Discord Domain in DNS Lookup (discord .com) alert_signature_id:2.035465e+06 app_proto:dns direction:to_server dns_query_0_id:44897 dns_query_0_opcode:0 dns_query_0_rrname:discord.com dns_query_0_rrtype:A dns_query_0_tx_id:8 dns_query_0_type:query flow_bytes_toclient:466 flow_bytes_toserver:395 flow_dest_ip:10.50.60.1 flow_dest_port:53 flow_id:3.99194650574119e+14 flow_pkts_toclient:4 flow_pkts_toserver:5 flow_src_ip:10.50.60.100 flow_src_port:65190 flow_start:2024-02-26T01:43:05.682768+0100 in_iface:bce1 pkt_src:wire/pcap proto:UDP timestamp:2024-02-26T01:43:33.514491+0100 tx_id:8] 1708908213514904703]"

meaning the plugin doesn't find the field "src_ip" for some reason.

[iShark5060]

ok, I was finally able to figure it out. Apparently the "src_ip" in my metrics are not fields, but tags. Had to convert them before passing them to the plugin. Works now.

[[processors.converter]]
  [processors.converter.tags]
    namepass = ["*suricata*"]
    order = 1
    string = ["src_ip"]

[[processors.converter]]
  [processors.converter.tags]
    namepass = ["*suricata*"]
    order = 2
    string = ["dest_ip"]