search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.38k stars 1.26k forks source link

ZEBRA: fpm_write: connection failed: Connection refused #8238

Open ghost opened 3 years ago

ghost commented 3 years ago

I can't attach zebra to fpm throuth tcp . Can you please help? 

The fpm implementation attempts to connect to 127.0.0.1 port 2620 by default without configurations.The dplane_fpm_nl only attempts to connect to a server if configured

netstat or ss doesnt show any tcp connections (1234 or default 2620) also sh zebra fpm stats show me zero route counters any examples how does fpm + zebra work?

( -M fpm:netlink or -M fpm:protobuf, or -M dplane_fpm_nl)

zebra 7.7-dev-eugene-gddbc1b4dd Ubuntu 18.04.5 LTS \n \l

ls /usr/lib/frr/modules/
bgpd_bmp.la   bgpd_snmp.la      ospf6d_snmp.la  ripd_snmp.la           zebra_fpm.la   zebra_snmp.la
bgpd_bmp.so   bgpd_snmp.so      ospf6d_snmp.so  ripd_snmp.so           zebra_fpm.so   zebra_snmp.so
bgpd_rpki.la  dplane_fpm_nl.la  ospfd_snmp.la   zebra_cumulus_mlag.la  zebra_irdp.la
bgpd_rpki.so  dplane_fpm_nl.so  ospfd_snmp.so   zebra_cumulus_mlag.so  zebra_irdp.so

sudo systemctl status frr
● frr.service - FRRouting
   Loaded: loaded (/etc/systemd/system/frr.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-03-11 14:14:10 MSK; 10min ago
     Docs: https://frrouting.readthedocs.io/en/latest/setup.html
  Process: 34448 ExecStop=/usr/lib/frr/frrinit.sh stop (code=exited, status=0/SUCCESS)
  Process: 34489 ExecStart=/usr/lib/frr/frrinit.sh start (code=exited, status=0/SUCCESS)
 Main PID: 34517 (watchfrr)
   Status: "FRR Operational"
    Tasks: 16 (limit: 9830)
   CGroup: /system.slice/frr.service
           ├─34517 /usr/lib/frr/watchfrr -d -F traditional zebra bgpd ospfd staticd
           ├─34540 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000 -M dplane_fpm_nl --log-level inf
           ├─34548 /usr/lib/frr/bgpd -d -F traditional -A 127.0.0.1
           ├─34557 /usr/lib/frr/ospfd -d -F traditional -A 127.0.0.1 --log file:/var/log/ospf.log --log-level deb
           └─34562 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1

./configure \
    --prefix=/usr \
    --includedir=\${prefix}/include \
    --enable-exampledir=\${prefix}/share/doc/frr/examples \
    --bindir=\${prefix}/bin \
    --sbindir=\${prefix}/lib/frr \
    --libdir=\${prefix}/lib/frr \
    --libexecdir=\${prefix}/lib/frr \
    --localstatedir=/var/run/frr \
    --sysconfdir=/etc/frr \
    --with-moduledir=\${prefix}/lib/frr/modules \
    --with-libyang-pluginsdir=\${prefix}/lib/frr/libyang_plugins \
    --enable-configfile-mask=0640 \
    --enable-logfile-mask=0640 \
    --enable-snmp=agentx \
    --enable-fpm \
    --enable-protobuf \
    --enable-multipath=64 \
    --enable-user=frr \
    --enable-group=frr \
    --enable-vty-group=frrvty \
    --enable-systemd=yes \
    --enable-rpki=yes \
    --with-pkg-git-version \
    --with-pkg-extra-version=-eugene

2021/03/11 13:51:42 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "condition" not found (./condition = 'ipv4-prefix-length' or
./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "action" not found (./action) with context node "/frr-route-map:lib/route-map/entry/set-action".
2021/03/11 13:51:42 ZEBRA: libyang: Schema node "action" not found (./action) with context node "/frr-route-map:lib/route-map/entry/set-action".
2021/03/11 13:51:42 ZEBRA: Setting netlink socket receive buffer size: 212992 -> 180000000
2021/03/11 13:51:42 ZEBRA: Setting netlink socket receive buffer size: 212992 -> 180000000
2021/03/11 13:51:42 ZEBRA: Setting netlink socket receive buffer size: 212992 -> 180000000
2021/03/11 13:51:42 ZEBRA: [EC 4043309110] Disabling MPLS support (no kernel support)
2021/03/11 13:51:42 ZEBRA: zebra 7.7-dev-eugene-gddbc1b4dd starting: vty@2601
2021/03/11 13:51:42 ZEBRA: Configuration Read in Took: 00:00:00
2021/03/11 13:51:42 ZEBRA: client 17 says hello and bids fair to announce only bgp routes vrf=0
2021/03/11 13:51:42 ZEBRA: client 35 says hello and bids fair to announce only vnc routes vrf=0
2021/03/11 13:51:42 OSPF: RI (ospf_router_info_init): Initialize Router Information
2021/03/11 13:51:42 OSPF: EXT (ospf_ext_init): Register Extended Link Opaque LSA
2021/03/11 13:51:42 OSPF: EXT (ospf_ext_init): Register Extended Prefix Opaque LSA
2021/03/11 13:51:42 OSPF: ospfd 7.7-dev-eugene-gddbc1b4dd starting: vty@2604
2021/03/11 13:51:42 ZEBRA: client 42 says hello and bids fair to announce only ospf routes vrf=0
2021/03/11 13:51:42 OSPF: Configuration Read in Took: 00:00:00
2021/03/11 13:51:42 ZEBRA: client 47 says hello and bids fair to announce only static routes vrf=0
2021/03/11 13:52:13 ZEBRA: fpm_process_event: manual FPM reconnect event
2021/03/11 13:52:16 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:19 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:22 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:25 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:28 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:31 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:34 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:37 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:40 ZEBRA: fpm_write: connection failed: Connection refused
2021/03/11 13:52:43 ZEBRA: fpm_write: connection failed: Connection refused
(config)# fpm address 127.0.0.1 port 1234
(config)# end
# sh fpm counters 
                  FPM counters
                  ============
                 Input bytes: 0
                Output bytes: 0
  Output buffer current size: 0
     Output buffer peak size: 0
           Connection closes: 0
           Connection errors: 27
  Data plane items processed: 0
   Data plane items enqueued: 0
 Data plane items queue peak: 0
            Buffer full hits: 0
     User FPM configurations: 2
   User FPM disable requests: 0

Also same issue with gentoo package net-misc/frr with fpm flag enabled

user@test ~ $ sudo netstat -tulpen | grep zebra
tcp        0      0 127.0.0.1:2601          0.0.0.0:*               LISTEN      371        66876      23390/zebra

as you see, zero counters everywhere.

user@test ~ $ sudo vtysh

Hello, this is FRRouting (version 7.5-gentoo).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

test# sh zebra fpm stats 

Counter                                       Total     Last 10 secs

connect_calls                                   284                2
connect_no_sock                                   0                0
read_cb_calls                                   284                2
write_cb_calls                                    0                0
write_calls                                       0                0
partial_writes                                    0                0
max_writes_hit                                    0                0
t_write_yields                                    0                0
nop_deletes_skipped                               0                0
route_adds                                        0                0
route_dels                                        0                0
updates_triggered                                 0                0
redundant_triggers                                0                0
dests_del_after_update                            0                0
t_conn_down_starts                                0                0
t_conn_down_dests_processed                       0                0
t_conn_down_yields                                0                0
t_conn_down_finishes                              0                0
t_conn_up_starts                                  0                0
t_conn_up_dests_processed                         0                0
t_conn_up_yields                                  0                0
t_conn_up_aborts                                  0                0
t_conn_up_finishes                                0                0
test# 

Mar 12 13:17:32 R1 zebra[993]: fpm_connect: attempting to connect to 127.0.0.1:2620
Mar 12 13:17:32 R1 zebra[993]: fpm_write: connection failed: Connection refused
Mar 12 13:17:35 R1 zebra[993]: zebra2proto: Please add this protocol(1) to proper rt_netlink.c handling
Mar 12 13:17:35 R1 zebra[993]: zebra2proto: Please add this protocol(1) to proper rt_netlink.c handling
Mar 12 13:17:35 R1 zebra[993]: zebra2proto: Please add this protocol(2) to proper rt_netlink.c handling
Mar 12 13:17:35 R1 zebra[993]: zebra2proto: Please add this protocol(2) to proper rt_netlink.c handling
Mar 12 13:17:35 R1 zebra[993]: fpm_connect: attempting to connect to 127.0.0.1:2620
Mar 12 13:17:35 R1 zebra[993]: fpm_process_event: next hop groups walk finished
Mar 12 13:17:35 R1 zebra[993]: fpm_process_event: RIB walk finished
Mar 12 13:17:35 R1 zebra[993]: fpm_process_event: RMAC walk finished
Mar 12 13:17:35 R1 zebra[993]: fpm_write: connection failed: Connection refused
ghost commented 3 years ago

also ospf routes missing in routing table (iproute2)

jmrcpn commented 3 years ago

Could this be related to linux kernel 5.12.+ Got a problem very similar.. ZEBRA: libyang: Schema node "condition" not found (./condition = 'ipv4-prefix-length' or ./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition".

compilation and execution context equal, problem show up with (new) kernel 5.12.3

Is there any progress on understanding that zebra trouble??

jmrcpn commented 3 years ago

My mistake (not related to kernel, didn't noticed libyang version was changed to 1.0.255) problem show with libyang version 1.0.215 (still show up with 1.0.255) zebra complains about.... Starting zebra daemon...2021/05/14 06:41:50 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "condition" not found (./condition = 'ipv4-prefix-length' or ./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "condition" not found (./condition) with context node "/frr-route-map:lib/route-map/entry/match-condition". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "action" not found (./action) with context node "/frr-route-map:lib/route-map/entry/set-action". 2021/05/14 06:41:50 ZEBRA: libyang: Schema node "action" not found (./action) with context node "/frr-route-map:lib/route-map/entry/set-action".

Back tracking to libyang-1.0.184

ziegenbalg commented 1 year ago

I'm getting this error as well. Using libyang1 (1.0.255) and frr (7.6). Could someone explain what the getsockopt call here is supposed to do and why it would fail? Thank you.

https://github.com/FRRouting/frr/blob/master/zebra/dplane_fpm_nl.c#L517