influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.11k stars 5.51k forks source link

Input plugin - Netflow incorrect ip_dscp field mapping for NSEL #14903

Closed SirBreadc closed 4 months ago

SirBreadc commented 4 months ago

Relevant telegraf.conf

[[inputs.netflow]]
  service_address = "udp4://:2055"

  ## Protocol version to use for decoding.
  ## Available options are
  ##   "ipfix"      -- IPFIX / Netflow v10 protocol (also works for Netflow v9)
  ##   "netflow v5" -- Netflow v5 protocol
  ##   "netflow v9" -- Netflow v9 protocol (also works for IPFIX)
  ##   "sflow v5"   -- sFlow v5 protocol
  protocol = "ipfix"

  dump_packets = false
# # Configuration for sending metrics to InfluxDB 2.0
 [[outputs.influxdb_v2]]
#   ## The URLs of the InfluxDB cluster nodes.
#   ##
#   ## Multiple URLs can be specified for a single cluster, only ONE of the
#   ## urls will be written to each interval.
#   ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
   urls = ["https://"]
#
#   ## Token for authentication.
   token = ""
#
#   ## Organization is the name of the organization you wish to write to.
   organization = ""
#
#   ## Destination bucket to write into.
   bucket = ""

Logs from Telegraf

2024-02-28T02:50:19Z D! [inputs.netflow] unknown Netflow v9 data field {false 299 0 [0 0 0 0 0 0 0 1]}
2024-02-28T02:50:19Z D! [inputs.netflow] unknown Netflow v9 data field {false 298 0 [0 0 0 0 0 0 0 1]}
2024-02-28T02:50:19Z D! [inputs.netflow] unknown Netflow v9 data field {false 299 0 [0 0 0 0 0 0 0 1]}
2024-02-28T02:50:19Z D! [inputs.netflow] unknown Netflow v9 data field {false 298 0 [0 0 0 0 0 0 0 1]}
2024-02-28T02:50:19Z D! [inputs.netflow] unknown Netflow v9 data field {false 299 0 [0 0 0 0 0 0 0 1]}

System info

Telegraf 1.30, Centos,CentOS Linux 7

Docker

No response

Steps to reproduce

1.Send NSEL from an FTD device with.

Expected behavior

initiator_packets and responder_packets are decoded

Actual behavior

initiator_packets and responder_packets fields are showing up as type_298 and type_299

Additional info

I tested a fix locally and by moving

    298: {{"initiator_packets", decodeUint}},               // initiatorPackets
    299: {{"responder_packets", decodeUint}},               // responderPackets

from var fieldMappingsIPFIX = map[uint16][]fieldMapping{ to var fieldMappingsNetflowCommon = map[uint16][]fieldMapping{ in telegraf/plugins/inputs/netflow/netflow_decoder.go to fix the issue. :)

Can anyone suggest other ways we could go about handling this?

srebhan commented 4 months ago

@SirBreadc see my comment in #14902. It seems like your device sends IPFIX data encapsulated in Netflow v9 packets...

srebhan commented 4 months ago

@SirBreadc can you please test the binary in #14910 available after CI finished all tests successfully? Let me know if this fixes the issue!

SirBreadc commented 4 months ago

@srebhan Yep that fixed my issue :) and is a much better fix that was I had suggested above image

srebhan commented 4 months ago

@SirBreadc what is the image supposed to tell me? ;-)

SirBreadc commented 4 months ago

haha sorry was in a hurry and just screenshotted the output in influx, but I can now see dscp value showing for both v9 and ipfix :) and the Version is being correctly marked. Before everything was showing as NetFlowV9 :) So the fix worked. How long does it normally take for fixed to be pushed up into a release build?

powersj commented 4 months ago

Next release is on or around March 10. You can use a nightly build until then once this is merged.