path selection in bgp is not updated accordingly to rpki updates

TrefoIV commented 1 month ago

Description

Hello,

I want that updates by the RPKI cache servers are applied to the adj-rib-in stored data and that stored announcements which become valid from an invalid state are considered in the best path selection process and inserted into the RIB.

In the documentation, I found the line: "Updates from the RPKI cache servers are directly applied and path selection is updated accordingly. (Soft reconfiguration must be enabled for this to work)" in the section "Feature of the Current Implementation". However, when a new certificate makes an announcement valid (from invalid state) the adj-rib-in is correctly processed but the route is not installed in the RIB unless the peering is manually cleared with "clear bgp neighbor x.x.x.x soft in".

Am I missing something? Or have I incorrectly interpreted the meaning of the above sentence?

Version

10.0.1

How to reproduce

Using two routers setup a peering between two differen ASes.
Setup a certification authority (I used krill to setup it)
router1 announces a route, call it x.x.x.x/y with AS number A
router2 implements rpki filters: valid routes are filtered in with high local-pref, notfound routes are filtered in with low local-pref and invalid routes are denyed. (The example route-map in the documentation would work as well).
there are two certificates for the route announced: "x.x.x.x/y => A" and "x.x.x.x/y => B"
verify that the route is correctly installed in router2's RIB and stored in the adj-rib-in

The configuration of router2 is the following. The rpki cache is located on the router itself, which runs "routinator".

!
!  BGP CONFIGURATION
!
debug bgp keepalives
debug bgp updates in
debug bgp updates out
dump bgp all-et /shared/dumps/dump-r2
!
router bgp C
    no bgp ebgp-requires-policy
    neighbor a.a.a.a remote-as A
    neighbor a.a.a.a  route-map rpki in
    neighbor a.a.a.a  soft-reconfiguration inbound
!
!
! RPKI
!
rpki
rpki polling_period 10
rpki cache 127.0.0.1 3323 pref 1
!
route-map rpki permit 4
match rpki valid
set local-preference 150
!
route-map rpki permit 6
match rpki notfound
set local-preference 10
!
route-map rpki deny 8
match rpki invalid
!
route-map rpki permit 40

remove the certificate "x.x.x.x/y => A". The route announced by router1 is now invalid
verify that router2 removed the route from the RIB, and that in the adj-rib-in from router1 the value of the local pref is now absent
restore the certificate "x.x.x.x/y => A". The route is now valid again.
verify that for router2, the route stored in the adj-rib-in from router1 has been processed again (the local pref is now set to the value associated to valid routes)
verify that the route IS NOT present in the router2's RIB.

The following zip folder three-line-peering.zip contains a [Kathará] lab that can be used to replicate the issue in an emulated environment on docker containers. The docker images used by "router2" (kathara/routinator) and "krill" (kathara/krill) machines can be build from dockerfile.zip. Other images are available on docker hub.

Expected behavior

When the route from router1 becomes valid again, router2 should select it again as best path and insert it into the RIB

Actual behavior

The route stored in the adj-rib-in is correctly processed by the rpki route-map but not selected

Additional context

No response

Checklist

[X] I have searched the open issues for this bug.
[X] I have not included sensitive information in this report.

ton31337 commented 1 month ago

Would you be able to test this patch https://github.com/FRRouting/frr/pull/16483?

TrefoIV commented 1 month ago

Hi Donatas,

sorry for the late reply, they've been some busy days.

I'm currently trying to test the patch you provided. I'm trying to build the patch in order to use in on a docker image for Kathara, bur I'm having trouble adding the rtrlib. In particular, I added the option -enable-rpki after the command "./configure" before building it, but it show the error message

" 16.79 configure: error: rtrlib was not found on your system or is too old. "

What should I do? Should I download the source code for rtrlib and put it somewhere? I can't understand it by looking at the error and at the documentation

Thanks, Samuele

Da: Donatas Abraitis @.> Inviato: sabato 27 luglio 2024 16:14 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Would you be able to test this patch #16483https://github.com/FRRouting/frr/pull/16483?

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2254160914, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AKBU2Z77D3CXHXGONDZOOTLVAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJUGE3DAOJRGQ. You are receiving this because you authored the thread.Message ID: @.***>

ton31337 commented 1 month ago

You can get the rpm/deb from here https://ci1.netdef.org/browse/FRR-PULLREQ3-4323/artifact.

TrefoIV commented 1 month ago

When trying to install from the .deb file i get an error due to dependency version mismatch:

the built version of frr is 9.0.1 but the package require version 10.2. The output is the following:

Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package frr-rpki-rtrlib.deb E: Couldn't find any package by glob 'frr-rpki-rtrlib.deb' @.***:/# apt install /frr-rpki-rtrlib.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done You might want to run 'apt --fix-broken install' to correct these. The following packages have unmet dependencies: frr-rpki-rtrlib : Depends: frr (= 10.2-dev-PR16483-gb1d2d52-20240727.135427-1~deb12u1) but it is not installed Depends: librtr0 but it is not installed E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

I'm sorry for the problems, but it is the first time I'm trying to build someone library from source

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 14:02 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

You can get the rpm/deb from here https://ci1.netdef.org/browse/FRR-PULLREQ3-4323/artifact.

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258182452, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AK7MXELRLWENZOMCUDZO56E3AVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGE4DENBVGI. You are receiving this because you authored the thread.Message ID: @.***>

ton31337 commented 1 month ago

Take librtr from here: https://ci1.netdef.org/artifact/RPKI-RTRLIB/shared/build-149.

TrefoIV commented 1 month ago

I downloaded the package .deb file at https://ci1.netdef.org/artifact/RPKI-RTRLIB/shared/build-149/Debian-11-x86_64-Packages/librtr-dev_0.8.0_amd64.deb but I get the same version error.

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 14:50 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Take librtr from here: https://ci1.netdef.org/artifact/RPKI-RTRLIB/shared/build-149.

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258275369, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3APQY6PAXNUMNOPBPLLZO6D27AVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGI3TKMZWHE. You are receiving this because you authored the thread.Message ID: @.***>

ton31337 commented 1 month ago

You don't need to compile. Just download librtr, install it, and then download the deb from https://ci1.netdef.org/browse/FRR-PULLREQ3-4323/artifact.

TrefoIV commented 1 month ago

I'm not compiling librtr, I'm compiling only the patched version of frr. Or am I missing something and don't need to compile it neither?

From: Donatas Abraitis @.> Sent: Tuesday, July 30, 2024 4:32:00 PM To: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Subject: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

You don't need to compile. Just download librtr, install it, and then download the deb from https://ci1.netdef.org/browse/FRR-PULLREQ3-4323/artifact.

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258502849, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3ANY4DHYSKF2USCEKX3ZO6PWBAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGUYDEOBUHE. You are receiving this because you authored the thread.Message ID: @.***>

ton31337 commented 1 month ago

You don't need to compile anything. Just install those two debs :)

TrefoIV commented 1 month ago

Ahh, I see.

Then, I downloaded and tried to install frr with the file https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1%7Edeb12u1_amd64.deb https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1~deb12u1_amd64.deb

Anyway, the problems are not finished 😭😭 I'm installing it using command "apt install ./filename.deb"

It show a dependency not found error:

The following packages have unmet dependencies: frr:i386 : Depends: lsof:i386 but it is not installable Depends: libc-ares2:i386 (>= 1.7.0) but it is not installable Depends: libc6:i386 (>= 2.34) but it is not installable Depends: libcap2:i386 (>= 1:2.10) but it is not installable Depends: libcrypt1:i386 (>= 1:4.1.0) but it is not installable Depends: libjson-c5:i386 (>= 0.15) but it is not installable Depends: liblua5.3-0:i386 but it is not installable Depends: libpam0g:i386 (>= 0.99.7.1) but it is not installable Depends: libprotobuf-c1:i386 (>= 1.0.1) but it is not installable Depends: libreadline8:i386 (>= 7.0~rc1) but it is not installable Depends: libyang2:i386 (>= 2.1.128~) but it is not installable

However, all this packages are correctly installed on my system (which is a Debian 12 docker container). In fact, trying to manually install them I get the output:

@.***:/# apt install lsof libc-ares2 libc6 libcap2 libcrypt1 libjson-c5 liblua5.3-0 libpam0g libprotobuf-c1 libreadline8 libyang2 Reading package lists... Done Building dependency tree... Done Reading state information... Done lsof is already the newest version (4.95.0-1). libc-ares2 is already the newest version (1.18.1-3). libc6 is already the newest version (2.36-9+deb12u7). libcap2 is already the newest version (1:2.66-4). libcrypt1 is already the newest version (1:4.4.33-2). libjson-c5 is already the newest version (0.16-2). liblua5.3-0 is already the newest version (5.3.6-2). libpam0g is already the newest version (1.5.2-6+deb12u1). libprotobuf-c1 is already the newest version (1.4.1-1+b1). libreadline8 is already the newest version (8.2-1.3). libyang2 is already the newest version (2.1.30-2).

Any idea why? 🙈🙈

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 17:08 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

You don't need to compile anything. Just install those two debs :)

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258585396, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AN26YBRTCPLSUDQVITZO6T7XAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGU4DKMZZGY. You are receiving this because you authored the thread.Message ID: @.***>

TrefoIV commented 1 month ago

Rectification.

I downloaded the wrong file for my platform.

Downloading the correct file it reduce the dependency error to just "libyang2" package:

frr : Depends: libyang2 (>= 2.1.128~) but it is not going to be installed

However, using apt, it install libyang2 version 2.1.30-2 saying it is the newest version.

Da: Samuele Quinzi @.> Inviato: martedì 30 luglio 2024 17:48 A: FRRouting/frr @.> Oggetto: R: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Ahh, I see.

Then, I downloaded and tried to install frr with the file https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1%7Edeb12u1_amd64.deb https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1~deb12u1_amd64.deb

Anyway, the problems are not finished 😭😭 I'm installing it using command "apt install ./filename.deb"

It show a dependency not found error:

The following packages have unmet dependencies: frr:i386 : Depends: lsof:i386 but it is not installable Depends: libc-ares2:i386 (>= 1.7.0) but it is not installable Depends: libc6:i386 (>= 2.34) but it is not installable Depends: libcap2:i386 (>= 1:2.10) but it is not installable Depends: libcrypt1:i386 (>= 1:4.1.0) but it is not installable Depends: libjson-c5:i386 (>= 0.15) but it is not installable Depends: liblua5.3-0:i386 but it is not installable Depends: libpam0g:i386 (>= 0.99.7.1) but it is not installable Depends: libprotobuf-c1:i386 (>= 1.0.1) but it is not installable Depends: libreadline8:i386 (>= 7.0~rc1) but it is not installable Depends: libyang2:i386 (>= 2.1.128~) but it is not installable

However, all this packages are correctly installed on my system (which is a Debian 12 docker container). In fact, trying to manually install them I get the output:

@.***:/# apt install lsof libc-ares2 libc6 libcap2 libcrypt1 libjson-c5 liblua5.3-0 libpam0g libprotobuf-c1 libreadline8 libyang2 Reading package lists... Done Building dependency tree... Done Reading state information... Done lsof is already the newest version (4.95.0-1). libc-ares2 is already the newest version (1.18.1-3). libc6 is already the newest version (2.36-9+deb12u7). libcap2 is already the newest version (1:2.66-4). libcrypt1 is already the newest version (1:4.4.33-2). libjson-c5 is already the newest version (0.16-2). liblua5.3-0 is already the newest version (5.3.6-2). libpam0g is already the newest version (1.5.2-6+deb12u1). libprotobuf-c1 is already the newest version (1.4.1-1+b1). libreadline8 is already the newest version (8.2-1.3). libyang2 is already the newest version (2.1.30-2).

Any idea why? 🙈🙈

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 17:08 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

You don't need to compile anything. Just install those two debs :)

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258585396, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AN26YBRTCPLSUDQVITZO6T7XAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGU4DKMZZGY. You are receiving this because you authored the thread.Message ID: @.***>

TrefoIV commented 1 month ago

I managed to find an installation of libyang2 with the correct version.

It seems I've installed all correctly 🎊🎊

I will add it to my Kathara lab and test if it fixes the initial bug. I'll keep you update

Da: Samuele Quinzi @.> Inviato: martedì 30 luglio 2024 18:05 A: FRRouting/frr @.> Oggetto: R: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Rectification.

I downloaded the wrong file for my platform.

Downloading the correct file it reduce the dependency error to just "libyang2" package:

frr : Depends: libyang2 (>= 2.1.128~) but it is not going to be installed

However, using apt, it install libyang2 version 2.1.30-2 saying it is the newest version.

Da: Samuele Quinzi @.> Inviato: martedì 30 luglio 2024 17:48 A: FRRouting/frr @.> Oggetto: R: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Ahh, I see.

Then, I downloaded and tried to install frr with the file https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1%7Edeb12u1_amd64.deb https://ci1.netdef.org/artifact/FRR-PULLREQ3/shared/build-4323/Debian-12-(Bookworm)-x86_64-Packages/frr_10.2-dev-PR16483-gb1d2d52-20240727.135427-1~deb12u1_amd64.deb

Anyway, the problems are not finished 😭😭 I'm installing it using command "apt install ./filename.deb"

It show a dependency not found error:

The following packages have unmet dependencies: frr:i386 : Depends: lsof:i386 but it is not installable Depends: libc-ares2:i386 (>= 1.7.0) but it is not installable Depends: libc6:i386 (>= 2.34) but it is not installable Depends: libcap2:i386 (>= 1:2.10) but it is not installable Depends: libcrypt1:i386 (>= 1:4.1.0) but it is not installable Depends: libjson-c5:i386 (>= 0.15) but it is not installable Depends: liblua5.3-0:i386 but it is not installable Depends: libpam0g:i386 (>= 0.99.7.1) but it is not installable Depends: libprotobuf-c1:i386 (>= 1.0.1) but it is not installable Depends: libreadline8:i386 (>= 7.0~rc1) but it is not installable Depends: libyang2:i386 (>= 2.1.128~) but it is not installable

However, all this packages are correctly installed on my system (which is a Debian 12 docker container). In fact, trying to manually install them I get the output:

@.***:/# apt install lsof libc-ares2 libc6 libcap2 libcrypt1 libjson-c5 liblua5.3-0 libpam0g libprotobuf-c1 libreadline8 libyang2 Reading package lists... Done Building dependency tree... Done Reading state information... Done lsof is already the newest version (4.95.0-1). libc-ares2 is already the newest version (1.18.1-3). libc6 is already the newest version (2.36-9+deb12u7). libcap2 is already the newest version (1:2.66-4). libcrypt1 is already the newest version (1:4.4.33-2). libjson-c5 is already the newest version (0.16-2). liblua5.3-0 is already the newest version (5.3.6-2). libpam0g is already the newest version (1.5.2-6+deb12u1). libprotobuf-c1 is already the newest version (1.4.1-1+b1). libreadline8 is already the newest version (8.2-1.3). libyang2 is already the newest version (2.1.30-2).

Any idea why? 🙈🙈

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 17:08 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

You don't need to compile anything. Just install those two debs :)

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258585396, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AN26YBRTCPLSUDQVITZO6T7XAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYGU4DKMZZGY. You are receiving this because you authored the thread.Message ID: @.***>

ton31337 commented 1 month ago

Also, adjust the timer for revalidation to see the adj-rib-in faster, e.g.:

rpki
 rpki revalidate_interval 5

TrefoIV commented 1 month ago

Problem fixed!!

Thank you very much for your support!

Da: Donatas Abraitis @.> Inviato: martedì 30 luglio 2024 18:37 A: FRRouting/frr @.> Cc: Samuele Quinzi @.>; Author @.> Oggetto: Re: [FRRouting/frr] path selection in bgp is not updated accordingly to rpki updates (Issue #16474)

Also, adjust the timer for revalidation to see the adj-rib-in faster, e.g.:

rpki rpki revalidate_interval 5

— Reply to this email directly, view it on GitHubhttps://github.com/FRRouting/frr/issues/16474#issuecomment-2258765860, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANZG3AOVHVXC3DR5GKPB24DZO66LHAVCNFSM6AAAAABLOVRFBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJYG43DKOBWGA. You are receiving this because you authored the thread.Message ID: @.***>

FRRouting / frr