search

codeout / fluent-plugin-sflow

sFlow plugin for Fluentd
MIT License
11 stars 3 forks source link

readme

fluent-plugin-sflow

Test on Ubuntu

This branch is valid for Fluentd 0.14.x or later

See 0.12.x branch for Fluentd 0.12.x.

Overview

Fluentd input plugin that acts as sFlow v2/v4/v5 collector.

Including a ruby wrapper of sflowtool which processes sFlow datagrams to report in JSON format.

Features

fluent-plugin-sflow supports many packet formats below. Basically, it can process anything sflowtool can do. :sparkles:

See sflowtool document for more details.

Sample Type

Address Family

Extended Data Type

Counter Data Type

Requirement

Install

You need to install development tools beforehand.

:memo: If you see any problem on other platforms rather than Debian, Ubuntu, or CentOS, please open a new issue.

Debian Linux or Ubuntu Linux

$ sudo apt install -y gcc make

If you're using ruby installed by apt, install ruby-dev.
( Skip if your ruby is installed by rbenv )

$ sudo apt install -y ruby-dev

CentOS Linux

$ sudo dnf install -y redhat-rpm-config gcc make

If you're using ruby installed by dnf ( or yum ), install ruby-devel.
( Skip if your ruby is installed by rbenv )

$ sudo dnf install -y ruby-devel

fluent-plugin-sflow

$ gem install fluent-plugin-sflow

You can also use fluent-gem or td-agent-gem instead of gem command.

Configuration

<source>
  @type sflow
  tag example.sflow

  bind 0.0.0.0
  port 6343
</source>

bind

IP address on which this plugin will accept sFlow. (Default: '0.0.0.0')

port

UDP port number on which this plugin will accept sFlow. (Default: 6343)

Record Example

Flow Sample

{
  "datagram_source_ip": "10.1.2.1",
  "datagram_size": 240,
  "unix_seconds_utc": 1502113217,
  "datagram_version": 5,
  "agent_sub_id": 0,
  "agent": "10.1.2.1",
  "packet_sequence_no": 53,
  "sys_up_time": 3808877,
  "samples_in_packet": 1,
  "sample_type_tag": "0:1",
  "sample_type": "flow_sample",
  "sample_sequence_no": 44,
  "source_id": "0:513",
  "mean_skip_count": 5,
  "sample_pool": 225,
  "drop_events": 0,
  "input_port": 513,
  "output_port": 512,
  "flow_block_tag": "0:1002",
  "header_protocol": 1,
  "sampled_packet_size": 102,
  "stripped_bytes": 4,
  "header_len": 98,
  "header_bytes": "02-05-86-71-7D-03-08-00-27-12-99-09-08-00-45-00-00-54-B0-61-40-00-40-01-79-44-0A-01-02-02-01-00-04-01-08-00-DF-97-04-12-03-AD-D2-1A-87-59-00-00-00-00-F6-61-02-00-00-00-00-00-10-11-12-13-14-15-16-17-18-19-1A-1B-1C-1D-1E-1F-20-21-22-23-24-25-26-27-28-29-2A-2B-2C-2D-2E-2F-30-31-32-33-34-35-36-37",
  "dst_mac": "020586717d03",
  "src_mac": "080027129909",
  "ip_size": 84,
  "ip_tot_len": 84,
  "src_ip": "10.1.2.2",
  "dst_ip": "1.0.4.1",
  "ip_protocol": 1,
  "ip_tos": 0,
  "ip_ttl": 64,
  "ip_id": 25008,
  "icmp_type": 8,
  "icmp_code": 0,
  "in_vlan": 0,
  "in_priority": 0,
  "out_vlan": 0,
  "out_priority": 0,
  "next_hop": "202.249.2.169",
  "src_subnet_mask": 32,
  "dst_subnet_mask": 22
}

Counters Sample

{
  "datagram_source_ip": "10.1.2.1",
  "datagram_size": 204,
  "unix_seconds_utc": 1502113415,
  "datagram_version": 5,
  "agent_sub_id": 0,
  "agent": "10.1.2.1",
  "packet_sequence_no": 54,
  "sys_up_time": 3810403,
  "samples_in_packet": 1,
  "sample_type_tag": "0:2",
  "sample_type": "counters_sample",
  "sample_sequence_no": 5,
  "source_id": "0:512",
  "counter_block_tag": "0:2",
  "if_index": 512,
  "network_type": 6,
  "if_speed": 10000000000,
  "if_direction": 1,
  "if_status": 3,
  "if_in_octets": 0,
  "if_in_ucast_pkts": 8,
  "if_in_multicast_pkts": 121,
  "if_in_broadcast_pkts": 0,
  "if_in_discards": 0,
  "if_in_errors": 0,
  "if_in_unknown_protos": 0,
  "if_out_octets": 130454,
  "if_out_ucast_pkts": 1060,
  "if_out_multicast_pkts": 0,
  "if_out_broadcast_pkts": 95,
  "if_out_discards": 0,
  "if_out_errors": 0,
  "if_promiscuous_mode": 0,
  "dot3_stats_alignment_errors": 0,
  "dot3_stats_fcserrors": 0,
  "dot3_stats_single_collision_frames": 0,
  "dot3_stats_multiple_collision_frames": 0,
  "dot3_stats_sqetest_errors": 0,
  "dot3_stats_deferred_transmissions": 0,
  "dot3_stats_late_collisions": 0,
  "dot3_stats_excessive_collisions": 0,
  "dot3_stats_internal_mac_transmit_errors": 0,
  "dot3_stats_carrier_sense_errors": 0,
  "dot3_stats_frame_too_longs": 0,
  "dot3_stats_internal_mac_receive_errors": 0,
  "dot3_stats_symbol_errors": 0
}

Benchmark

Here is a quick benchmark on Macbook Pro Mid 2015:

TODO

Contributing

Please report issues or enhancement requests to GitHub issues. For questions or feedbacks write to my twitter @codeout.

Or send a pull request to fix.

Copyright and License

Copyright (c) 2017-2021 Shintaro Kojima. Code released under the MIT license.

Code includes a part of sflowtool which is distributed in the InMon sFlow License.