search

tynany / frr_exporter

Prometheus exporter for Free Range Routing
MIT License
100 stars 34 forks source link

Not all descriptions are filled up for non default vrf #74

Closed 33Fraise33 closed 2 years ago

33Fraise33 commented 2 years ago

Hello,

We notice that not all descriptions in our exporter are complete.

for example I have the result: frr_bgp_peer_message_received_total{afi="ipv4",local_as="65264",peer="10.189.0.178",peer_as="64822",peer_desc="",safi="unicast",vrf="nni"} 0

While if I run the following command the description is there: 

$ sudo vtysh -c "show bgp vrf all neighbors 10.189.0.178 json" | grep Desc
    "nbrDesc":"\"PR-KMDA\"",

Is this expected behaviour and can this be resolved?

frr version: 

show version
FRRouting 7.5+cl4.4.0u4 (DC-IX-CORESW02).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--enable-exampledir=/usr/share/doc/frr/examples/' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--disable-dev-build' '--enable-systemd=yes' '--enable-rpki' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-zeromq' '--enable-ospfapi' '--disable-bgp-vnc' '--enable-multipath=128' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' '--disable-address-sanitizer' '--enable-protobuf' '--enable-cumulus=yes' '--enable-csmgr=yes' '--enable-datacenter=yes' '--enable-bfdd=no' '--enable-sharpd=yes' '--enable-grpc=yes' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'
tynany commented 2 years ago

I am not able to replicate this behaviour.

Can you confirm you are running frr_exporter with the --collector.bgp.peer-descriptions and --collector.bgp.peer-descriptions.plain-text flag?

Below is how I've tried to recreate the issue:

# Start FRR Exporter with the --collector.bgp.peer-descriptions and --collector.bgp.peer-descriptions.plain-text flag:
$ sudo ./frr_exporter --collector.bgp.peer-descriptions --collector.bgp.peer-descriptions.plain-text &

# Confirm FRR Exporter metrics contain the description:
$ curl --silent localhost:9342/metrics | grep PR-KMDA
frr_bgp_peer_message_received_total{afi="ipv4",local_as="64800",peer="10.189.0.178",peer_as="0",peer_desc="PR-KMDA",safi="unicast",vrf="default"} 0
frr_bgp_peer_message_sent_total{afi="ipv4",local_as="64800",peer="10.189.0.178",peer_as="0",peer_desc="PR-KMDA",safi="unicast",vrf="default"} 0
frr_bgp_peer_prefixes_received_count_total{afi="ipv4",local_as="64800",peer="10.189.0.178",peer_as="0",peer_desc="PR-KMDA",safi="unicast",vrf="default"} 0
frr_bgp_peer_state{afi="ipv4",local_as="64800",peer="10.189.0.178",peer_as="0",peer_desc="PR-KMDA",safi="unicast",vrf="default"} 0
frr_bgp_peer_uptime_seconds{afi="ipv4",local_as="64800",peer="10.189.0.178",peer_as="0",peer_desc="PR-KMDA",safi="unicast",vrf="default"} 0

# Confirm FRR is exposing the description in the JSON output:
$ sudo vtysh -c "show bgp vrf all neighbors 10.189.0.178 json" | grep Desc
    "nbrDesc":"PR-KMDA",

# BGP config I'm using:
$ sudo vtysh -c "show run bgp" | grep "10.189.0.178"
 neighbor 10.189.0.178 remote-as external
 neighbor 10.189.0.178 description PR-KMDA

Can you please provide more info such as:

33Fraise33 commented 2 years ago

Hello,

Added is the current config of our exporter: 

$ cat /etc/systemd/system/frr_exporter@.service 
[Unit]
Description=Prometheus FRR Exporter
After=network-online.target

[Service]
Type=simple
User=root
Group=root
#ExecStart=/usr/local/bin/frr_exporter
ExecStart=/usr/local/bin/frr_exporter --frr.socket.dir-path="/var/run/frr" --no-collector.ospf --no-collector.bfd --collector.bgp6 --collector.bgpl2vpn --collector.bgp.peer-descriptions --collector.bgp.peer-descriptions.plain-text
SyslogIdentifier=frr_exporter
Restart=always
RestartSec=1
StartLimitInterval=0

[Install]
WantedBy=multi-user.target
gianni@DC-IX-CORESW02:mgmt:~$ sudo systemctl status frr_exporter@mgmt.service 
● frr_exporter@mgmt.service - Prometheus FRR Exporter
   Loaded: loaded (/etc/systemd/system/frr_exporter@.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2022-06-27 14:07:51 CEST; 1 weeks 1 days ago
 Main PID: 17956 (frr_exporter)
   Memory: 7.3M
   CGroup: /system.slice/system-frr_exporter.slice/frr_exporter@mgmt.service
           └─17956 /usr/local/bin/frr_exporter --frr.socket.dir-path=/var/run/frr --no-collector.ospf --no-collector.bfd --collector.bgp6 --collector.bgpl2vpn --collector.bgp.peer-descriptions --collector.bgp.peer-descriptions.plain-text

Jun 27 14:07:51 DC-IX-CORESW02 systemd[1]: Started Prometheus FRR Exporter.
Jun 27 14:07:51 DC-IX-CORESW02 frr_exporter[17956]: level=info ts=2022-06-27T12:07:51.641Z caller=frr_exporter.go:65 msg="Starting frr_exporter" version="(version=1.1.0, branch=HEAD, revision=5ac721e047f91e8906ece32ac2ff62797e2a5d6a)"
Jun 27 14:07:51 DC-IX-CORESW02 frr_exporter[17956]: level=info ts=2022-06-27T12:07:51.641Z caller=frr_exporter.go:66 msg="Build context" build_context="(go=go1.17.5, user=root@14669bca9510, date=20211221-20:17:31)"
Jun 27 14:07:51 DC-IX-CORESW02 frr_exporter[17956]: level=info ts=2022-06-27T12:07:51.641Z caller=frr_exporter.go:67 msg="Listening on address" address=:9342
Jun 27 14:07:51 DC-IX-CORESW02 frr_exporter[17956]: level=info ts=2022-06-27T12:07:51.642Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
giannistubbe@DC-IX-CORESW02:mgmt:~$

And yes some descriptions are working but only in the default VRF.

I have the same thing on my lab environment running the latest version of FRR and frr_exporter (the above example is frr on cumulus so not the latest version).

An output from the lab environment looks like this: 

# HELP frr_bgp_peer_groups_count_total Number of peer groups configured.
# TYPE frr_bgp_peer_groups_count_total gauge
frr_bgp_peer_groups_count_total{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 1
frr_bgp_peer_groups_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 1
frr_bgp_peer_groups_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_groups_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_groups_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 1
frr_bgp_peer_groups_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 1
# HELP frr_bgp_peer_groups_memory_bytes Memory consumed by peer groups.
# TYPE frr_bgp_peer_groups_memory_bytes gauge
frr_bgp_peer_groups_memory_bytes{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 64
frr_bgp_peer_groups_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 64
frr_bgp_peer_groups_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 64
frr_bgp_peer_groups_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 64
frr_bgp_peer_groups_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 64
frr_bgp_peer_groups_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 64
# HELP frr_bgp_peer_message_received_total Number of received messages.
# TYPE frr_bgp_peer_message_received_total counter
frr_bgp_peer_message_received_total{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 157736
frr_bgp_peer_message_received_total{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 157665
frr_bgp_peer_message_received_total{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 157271
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1.406259e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1.407226e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1.410277e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 83863
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 53917
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1.640587e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1.585025e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1.611501e+06
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 2.5389922e+07
frr_bgp_peer_message_received_total{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 157670
# HELP frr_bgp_peer_message_sent_total Number of sent messages.
# TYPE frr_bgp_peer_message_sent_total counter
frr_bgp_peer_message_sent_total{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 157985
frr_bgp_peer_message_sent_total{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 157798
frr_bgp_peer_message_sent_total{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 157411
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 157989
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 157987
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 157988
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 47381
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 47251
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 158064
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 158012
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 158036
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 158024
frr_bgp_peer_message_sent_total{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 157808
# HELP frr_bgp_peer_prefixes_advertised_count_total Number of prefixes advertised.
# TYPE frr_bgp_peer_prefixes_advertised_count_total gauge
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 9
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 9
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 9
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 1
frr_bgp_peer_prefixes_advertised_count_total{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 13
# HELP frr_bgp_peer_prefixes_received_count_total Number of prefixes received.
# TYPE frr_bgp_peer_prefixes_received_count_total gauge
frr_bgp_peer_prefixes_received_count_total{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 0
frr_bgp_peer_prefixes_received_count_total{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 4
frr_bgp_peer_prefixes_received_count_total{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 1
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 38025
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 38021
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 38021
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 603
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 38641
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 38641
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 38641
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 152624
frr_bgp_peer_prefixes_received_count_total{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 7
# HELP frr_bgp_peer_state State of the peer (2 = Administratively Down, 1 = Established, 0 = Down).
# TYPE frr_bgp_peer_state gauge
frr_bgp_peer_state{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 1
frr_bgp_peer_state{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 1
frr_bgp_peer_state{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 1
frr_bgp_peer_state{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 1
# HELP frr_bgp_peer_uptime_seconds How long has the peer been up.
# TYPE frr_bgp_peer_uptime_seconds gauge
frr_bgp_peer_uptime_seconds{afi="ipv4",local_as="211184",peer="10.10.34.10",peer_as="65502",peer_desc="",safi="unicast",vrf="remotesites"} 487574
frr_bgp_peer_uptime_seconds{afi="ipv4",local_as="211184",peer="10.10.34.2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 732381
frr_bgp_peer_uptime_seconds{afi="ipv4",local_as="211184",peer="10.10.34.6",peer_as="65511",peer_desc="",safi="unicast",vrf="remotesites"} 608244
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:1",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 734805
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:2",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 734800
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:33::a103:1142:3",peer_as="31142",peer_desc="",safi="unicast",vrf="kleyrex"} 734802
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::111",peer_as="47498",peer_desc="",safi="unicast",vrf="fogixp"} 31805
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:ca:1::77",peer_as="210387",peer_desc="",safi="unicast",vrf="fogixp"} 31802
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::babe:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 32841
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::be5a",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 32655
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2001:7f8:f2:e1::dead:1",peer_as="202409",peer_desc="",safi="unicast",vrf="locix"} 32831
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2a0c:9a40:1::1",peer_as="34927",peer_desc="IFOG PEERING",safi="unicast",vrf="default"} 734784
frr_bgp_peer_uptime_seconds{afi="ipv6",local_as="211184",peer="2a12:4946:4001:34::1:2",peer_as="65501",peer_desc="",safi="unicast",vrf="remotesites"} 732381
# HELP frr_bgp_peers_count_total Number peers configured.
# TYPE frr_bgp_peers_count_total gauge
frr_bgp_peers_count_total{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 3
frr_bgp_peers_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 1
frr_bgp_peers_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 2
frr_bgp_peers_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 3
frr_bgp_peers_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 3
frr_bgp_peers_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 1
# HELP frr_bgp_peers_memory_bytes Memory consumed by peers.
# TYPE frr_bgp_peers_memory_bytes gauge
frr_bgp_peers_memory_bytes{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 2.222424e+06
frr_bgp_peers_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 740808
frr_bgp_peers_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 1.481616e+06
frr_bgp_peers_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 2.222424e+06
frr_bgp_peers_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 2.222424e+06
frr_bgp_peers_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 740808
# HELP frr_bgp_rib_count_total Number of routes in the RIB.
# TYPE frr_bgp_rib_count_total gauge
frr_bgp_rib_count_total{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 19
frr_bgp_rib_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 288339
frr_bgp_rib_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 1172
frr_bgp_rib_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 73130
frr_bgp_rib_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 74135
frr_bgp_rib_count_total{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 23
# HELP frr_bgp_rib_memory_bytes Memory consumbed by the RIB.
# TYPE frr_bgp_rib_memory_bytes gauge
frr_bgp_rib_memory_bytes{afi="ipv4",local_as="211184",safi="unicast",vrf="remotesites"} 3496
frr_bgp_rib_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="default"} 5.3054376e+07
frr_bgp_rib_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="fogixp"} 215648
frr_bgp_rib_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="kleyrex"} 1.345592e+07
frr_bgp_rib_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="locix"} 1.364084e+07
frr_bgp_rib_memory_bytes{afi="ipv6",local_as="211184",safi="unicast",vrf="remotesites"} 4232
# HELP frr_collector_up Whether the collector's last scrape was successful (1 = successful, 0 = unsuccessful).
# TYPE frr_collector_up gauge
frr_collector_up{collector="bgp"} 1
frr_collector_up{collector="bgp6"} 1
# HELP frr_exporter_build_info A metric with a constant '1' value labeled by version, revision, branch, and goversion from which frr_exporter was built.
# TYPE frr_exporter_build_info gauge
frr_exporter_build_info{branch="HEAD",goversion="go1.17.9",revision="d0d5ec547513a0995543e1da6f44d18491d0e44f",version="1.1.1"} 1
# HELP frr_scrape_duration_seconds Time it took for a collector's scrape to complete.
# TYPE frr_scrape_duration_seconds gauge
frr_scrape_duration_seconds{collector="bgp"} 0.000597847
frr_scrape_duration_seconds{collector="bgp6"} 0.000967485
# HELP frr_scrapes_total Total number of times FRR has been scraped.
# TYPE frr_scrapes_total counter
frr_scrapes_total 51838
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 1.6679e-05
go_gc_duration_seconds{quantile="0.25"} 1.78e-05
go_gc_duration_seconds{quantile="0.5"} 1.8522e-05
go_gc_duration_seconds{quantile="0.75"} 2.431e-05
go_gc_duration_seconds{quantile="1"} 6.6905e-05
go_gc_duration_seconds_sum 0.146949924
go_gc_duration_seconds_count 6504
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 8
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.17.9"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 2.20384e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 1.7322900536e+10
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 4567
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 2.36345534e+08
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
go_memstats_gc_cpu_fraction 2.417102155813244e-06
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 5.1664e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 2.20384e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 5.16096e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 2.965504e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 14688
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 3.825664e+06
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 8.126464e+06
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.65709955165514e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 2.36360222e+08
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 1200
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 16384
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 39984
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 65536
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 4.194304e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 467961
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 262144
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 262144
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 1.4109456e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 4
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 115.86
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 10
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.256896e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.65631898049e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 7.30169344e+08
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19

and my frr config looks like this: 

!
frr version 8.2.2
frr defaults traditional
hostname kingpin
service integrated-vtysh-config
!
ipv6 route ::/0 2a0c:9a40:1::1 10
!
router bgp 211184
 bgp router-id 193.148.249.106
 no bgp default ipv4-unicast
 no bgp client-to-client reflection
 bgp graceful-restart
 no bgp network import-check
 neighbor default peer-group
 neighbor default timers 20 60
 neighbor 2a0c:9a40:1::1 remote-as 34927
 neighbor 2a0c:9a40:1::1 peer-group default
 neighbor 2a0c:9a40:1::1 description IFOG PEERING
 !
 address-family ipv4 unicast
  redistribute connected
  redistribute static
  neighbor default soft-reconfiguration inbound
  neighbor default route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor default route-map rm-AS211184-exact out
  import vrf remotesites
 exit-address-family
 !
 address-family ipv6 unicast
  aggregate-address 2a12:4946:4000::/44
  redistribute connected
  redistribute static
  neighbor default activate
  neighbor default soft-reconfiguration inbound
  neighbor default route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor default route-map rm-AS211184-exact out
  import vrf locix
  import vrf kleyrex
  import vrf remotesites
 exit-address-family
exit
!
router bgp 211184 vrf remotesites
 bgp router-id 10.10.33.1
 no bgp default ipv4-unicast
 no bgp client-to-client reflection
 no bgp network import-check
 neighbor remotesites peer-group
 neighbor remotesites timers 20 60
 neighbor 10.10.34.2 remote-as 65501
 neighbor 10.10.34.2 peer-group remotesites
 neighbor 10.10.34.2 description ASGARDv4
 neighbor 10.10.34.6 remote-as 65511
 neighbor 10.10.34.6 peer-group remotesites
 neighbor 10.10.34.6 description MOMv4
 neighbor 10.10.34.10 remote-as 65502
 neighbor 10.10.34.10 peer-group remotesites
 neighbor 10.10.34.10 description SAKAARv4
 neighbor 2a12:4946:4001:34::1:2 remote-as 65501
 neighbor 2a12:4946:4001:34::1:2 peer-group remotesites
 neighbor 2a12:4946:4001:34::1:2 description ASGARDv6
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor remotesites soft-reconfiguration inbound
  neighbor remotesites route-map rm-REMOTE_SITES-in in
  neighbor remotesites route-map rm-REMOTE_SITES-out out
  neighbor 10.10.34.2 activate
  neighbor 10.10.34.6 activate
  neighbor 10.10.34.10 activate
  import vrf default
 exit-address-family
 !
 address-family ipv6 unicast
  aggregate-address 2a12:4946:4000::/44
  redistribute connected
  neighbor remotesites soft-reconfiguration inbound
  neighbor remotesites route-map rm-REMOTE_SITES-in in
  neighbor remotesites route-map rm-REMOTE_SITES-out out
  neighbor 2a12:4946:4001:34::1:2 activate
  import vrf route-map rm-REMOTE_SITES-out
  import vrf default
 exit-address-family
exit
!
router bgp 211184 vrf locix
 bgp router-id 185.1.167.89
 no bgp default ipv4-unicast
 no bgp client-to-client reflection
 bgp graceful-restart
 no bgp network import-check
 neighbor locix peer-group
 neighbor locix remote-as 202409
 neighbor locix timers 20 60
 neighbor 2001:7f8:f2:e1::be5a peer-group locix
 neighbor 2001:7f8:f2:e1::be5a description Locix RS3
 neighbor 2001:7f8:f2:e1::babe:1 peer-group locix
 neighbor 2001:7f8:f2:e1::babe:1 description Locix RS1
 neighbor 2001:7f8:f2:e1::dead:1 peer-group locix
 neighbor 2001:7f8:f2:e1::dead:1 description Locix RS2
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor locix soft-reconfiguration inbound
  neighbor locix route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor locix route-map rm-AS211184-exact out
 exit-address-family
 !
 address-family ipv6 unicast
  network 2a12:4946:4000::/44
  redistribute connected
  neighbor locix activate
  neighbor locix soft-reconfiguration inbound
  neighbor locix route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor locix route-map rm-AS211184-exact out
  import vrf route-map rm-AS211184
  import vrf default
 exit-address-family
exit
!
router bgp 211184 vrf kleyrex
 bgp router-id 193.189.83.6
 no bgp default ipv4-unicast
 no bgp client-to-client reflection
 bgp graceful-restart
 no bgp network import-check
 neighbor kleyrex peer-group
 neighbor kleyrex remote-as 31142
 neighbor kleyrex timers 20 60
 neighbor 2001:7f8:33::a103:1142:1 peer-group kleyrex
 neighbor 2001:7f8:33::a103:1142:1 description Kleyrex RS1
 neighbor 2001:7f8:33::a103:1142:2 peer-group kleyrex
 neighbor 2001:7f8:33::a103:1142:2 description Kleyrex RS2
 neighbor 2001:7f8:33::a103:1142:3 peer-group kleyrex
 neighbor 2001:7f8:33::a103:1142:3 description Kleyrex RS3
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor kleyrex soft-reconfiguration inbound
  neighbor kleyrex route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor kleyrex route-map rm-AS211184-exact out
 exit-address-family
 !
 address-family ipv6 unicast
  network 2a12:4946:4000::/44
  redistribute connected
  neighbor kleyrex activate
  neighbor kleyrex soft-reconfiguration inbound
  neighbor kleyrex route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor kleyrex route-map rm-AS211184-exact out
  import vrf route-map rm-AS211184
  import vrf default
 exit-address-family
exit
!
router bgp 211184 vrf fogixp
 bgp router-id 185.1.147.45
 no bgp default ipv4-unicast
 no bgp client-to-client reflection
 bgp graceful-restart
 no bgp network import-check
 neighbor fogixp peer-group
 neighbor fogixp timers 20 60
 neighbor 2001:7f8:ca:1::77 remote-as 210387
 neighbor 2001:7f8:ca:1::77 peer-group fogixp
 neighbor 2001:7f8:ca:1::77 description Bubacarr Sowe
 neighbor 2001:7f8:ca:1::111 remote-as 47498
 neighbor 2001:7f8:ca:1::111 peer-group fogixp
 neighbor 2001:7f8:ca:1::111 description FogIXP RS1
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor fogixp soft-reconfiguration inbound
  neighbor fogixp route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor fogixp route-map rm-AS211184-exact out
 exit-address-family
 !
 address-family ipv6 unicast
  network 2a12:4946:4000::/44
  redistribute connected
  neighbor fogixp activate
  neighbor fogixp soft-reconfiguration inbound
  neighbor fogixp route-map rm-DEFAULT_ONLY_PUBLIC in
  neighbor fogixp route-map rm-AS211184-exact out
  import vrf route-map rm-AS211184
  import vrf default
 exit-address-family
exit
!
ip prefix-list pl-default-public seq 5 permit 0.0.0.0/0 ge 0 le 24
ip prefix-list pl-default seq 5 permit 0.0.0.0/0
ip prefix-list pl-RFC1918 seq 5 permit 10.0.0.0/8 ge 8
ip prefix-list pl-RFC1918 seq 10 permit 172.16.0.0/12 ge 12
ip prefix-list pl-RFC1918 seq 15 permit 192.168.0.0/16 ge 16
ip prefix-list pl-LocalIpv4s seq 5 permit 10.10.33.1/32
ip prefix-list pl-LocalIpv4s seq 10 permit 193.148.249.0/24
ip prefix-list pl-LocalIpv4s seq 15 permit 185.1.166.0/23
!
ipv6 prefix-list pl-default-public seq 5 permit ::/0 ge 0 le 48
ipv6 prefix-list pl-default seq 5 permit ::/0
ipv6 prefix-list pl-AS211184-exact seq 5 permit 2a12:4946:4000::/44
ipv6 prefix-list pl-AS211184 seq 5 permit 2a12:4946:4000::/44 ge 44
!
route-map rm-DEFAULT_ONLY_PUBLIC permit 10
 match ip address prefix-list pl-default-public
exit
!
route-map rm-DEFAULT_ONLY_PUBLIC permit 20
 match ipv6 address prefix-list pl-default-public
exit
!
route-map rm-AS211184-exact permit 10
 match ipv6 address prefix-list pl-AS211184-exact
exit
!
route-map rm-AS211184 permit 10
 match ipv6 address prefix-list pl-AS211184
exit
!
route-map rm-REMOTE_SITES-in permit 10
 match ipv6 address prefix-list pl-AS211184
exit
!
route-map rm-REMOTE_SITES-in permit 20
 match ip address prefix-list pl-RFC1918
exit
!
route-map rm-REMOTE_SITES-out permit 10
 match ipv6 address prefix-list pl-AS211184
exit
!
route-map rm-REMOTE_SITES-out permit 20
 match ip address prefix-list pl-RFC1918
exit
!
route-map rm-REMOTE_SITES-out permit 30
 match ip address prefix-list pl-default
exit
!
route-map rm-REMOTE_SITES-out permit 40
 match ipv6 address prefix-list pl-default
exit
!
route-map rm-LocalIpv4s permit 10
 match ip address prefix-list pl-LocalIpv4s
exit
!
route-map rm-default permit 10
 match ipv6 address prefix-list pl-default
exit
!
route-map rm-default permit 20
 match ip address prefix-list pl-default
exit
!
end
tynany commented 2 years ago

Thanks for the detailed information. I see the issue and I'll have a fix ready to go in a day or two.