neptune-networks / flow-exporter

Export network flows from Kafka to Prometheus
https://brooks.sh/2019/11/17/network-flow-analysis-with-prometheus/
MIT License
125 stars 20 forks source link

Help to run the flow #15

Open DuhBatista opened 2 years ago

DuhBatista commented 2 years ago

Hello colleagues.

I'm trying to lift collector. But I'm having a problem and some doubts with the points mentioned here.

1- kafka_broker_host: kafka-host.fqdn.com what can we put in this field in a simple and functional way could it be 127.0.0.1?

2- set_label=bdr1.neptunenetworks.org what we can also put in this field could be none ?

3- KAFKA_ADVERTISED_LISTENERS: INSIDE://:19092,OUTSIDE://kafka.fqdn.com:9092 at this point would the same indication 127.0.0.1 be relevant if that were the case?

Log in after the bgp connection is established. Sep 20 21:41:16 traffic-flow pmacctd[467]: INFO ( default/core/BGP ): [10.254.0.78] BGP connection reset by peer (0). Sep 20 21:41:16 traffic-flow pmacctd[467]: INFO ( default_kafka/kafka ): Purging cache - START (PID: 9631) Sep 20 21:41:17 traffic-flow pmacctd[467]: INFO ( default/core/BGP ): [10.10.30.250] BGP peers usage: 1/2 Sep 20 21:41:17 traffic-flow pmacctd[467]: INFO ( default/core/BGP ): [10.254.0.78] Capability: 4-bytes AS [41] ASN [26xx9] Sep 20 21:41:17 traffic-flow pmacctd[467]: INFO ( default/core/BGP ): [10.254.0.78] Capability: MultiProtocol [1] AFI [1] SAFI [1] Sep 20 21:41:17 traffic-flow pmacctd[467]: INFO ( default/core/BGP ): [10.254.0.78] BGP_OPEN: Local AS: 26xx9 Remote AS: 26xx9 HoldTime: 90 Sep 20 21:41:21 traffic-flow pmacctd[467]: INFO ( default_kafka/kafka ): Purging cache - START (PID: 9635) Sep 20 21:41:22 traffic-flow pmacctd[467]: ERROR ( default_kafka/kafka ): Connection failed to Kafka: p_kafka_check_outq_len() Sep 20 21:41:22 traffic-flow pmacctd[467]: ERROR ( default_kafka/kafka ): Connection failed to Kafka: p_kafka_close() Sep 20 21:41:22 traffic-flow pmacctd[467]: INFO ( default_kafka/kafka ): Purging cache - END (PID: 9631, QN: 1416/1416, ET: 0) Sep 20 21:41:26 traffic-flow pmacctd[467]: INFO ( default_kafka/kafka ): Purging cache - START (PID: 9639)

DuhBatista commented 2 years ago

My error was in /etc/pmacct/peering_agent.map The ip was wrong. Now I have an output equal to the one reported in the example after installation. Exit: {"event_type": "purge", "label": "10.10.30.250", "as_src": 26xx9, "as_dst": 0, "ip_src":

{"event_type": "purge", "label": "10.10.30.250", "as_src": 0, "as_dst": 26xx9, "ip_src":

I see the "zero" in evidence for as_src and as_dst even though I'm changing helping the bgp_agent_map.