search

opensourcerouting / frr

Free Range Routing Protocol Suite
Other
37 stars 12 forks source link

topotato: WIP test_bgp_orf.py #99

Closed Chromico closed 1 year ago

Chromico commented 1 year ago

( Work In Progress ) bgp_converge_r1 and bgp_converge_r2 work. However, bgp_orf_changed_r1 fails.

Here's the test result:

➜  basetato4 git:(bgp-orf) ✗ ./run_userns.sh --frr-builddir=/root/buildfrr/ --log-cli-level=DEBUG -v -v  -x test_bgp_orf.py
============================================================================== topotato initialization ==============================================================================

------------------------------------------------------------------------------- live log sessionstart -------------------------------------------------------------------------------
DEBUG    topotato:pretty.py:146 executable dot found: /usr/bin/dot
DEBUG    topotato:frr.py:149 FRR build directory: '/root/buildfrr'
DEBUG    topotato:frr.py:164 FRR source directory: '/root/buildfrr'
INFO     topotato:frr.py:203 FRR daemons: zebra, staticd, babeld, bfdd, bgpd, eigrpd, fabricd, isisd, ldpd, nhrpd, ospf6d, ospfd, pathd, pbrd, pim6d, pimd, ripd, ripngd, vrrpd
DEBUG    topotato:frr.py:215 zebra => zebra/zebra
DEBUG    topotato:frr.py:213 ignoring target 'watchfrr/watchfrr'
DEBUG    topotato:frr.py:213 ignoring target 'tools/ssd'
DEBUG    topotato:frr.py:215 bgpd => bgpd/bgpd
DEBUG    topotato:frr.py:215 ripd => ripd/ripd
DEBUG    topotato:frr.py:215 ripngd => ripngd/ripngd
DEBUG    topotato:frr.py:215 ospfd => ospfd/ospfd
DEBUG    topotato:frr.py:215 ospf6d => ospf6d/ospf6d
DEBUG    topotato:frr.py:215 isisd => isisd/isisd
DEBUG    topotato:frr.py:215 fabricd => isisd/fabricd
DEBUG    topotato:frr.py:215 nhrpd => nhrpd/nhrpd
DEBUG    topotato:frr.py:215 ldpd => ldpd/ldpd
DEBUG    topotato:frr.py:215 babeld => babeld/babeld
DEBUG    topotato:frr.py:215 eigrpd => eigrpd/eigrpd
DEBUG    topotato:frr.py:215 pimd => pimd/pimd
DEBUG    topotato:frr.py:215 pbrd => pbrd/pbrd
DEBUG    topotato:frr.py:215 staticd => staticd/staticd
DEBUG    topotato:frr.py:215 bfdd => bfdd/bfdd
DEBUG    topotato:frr.py:215 vrrpd => vrrpd/vrrpd
DEBUG    topotato:frr.py:215 pathd => pathd/pathd
DEBUG    topotato:frr.py:213 ignoring target 'lib/grammar_sandbox'
DEBUG    topotato:frr.py:213 ignoring target 'lib/clippy'
DEBUG    topotato:frr.py:213 ignoring target 'tools/permutations'
DEBUG    topotato:frr.py:213 ignoring target 'tools/gen_northbound_callbacks'
DEBUG    topotato:frr.py:213 ignoring target 'tools/gen_yang_deviations'
DEBUG    topotato:frr.py:213 ignoring target 'bgpd/bgp_btoa'
DEBUG    topotato:frr.py:213 ignoring target 'bgpd/rfp-example/rfptest/rfptest'
DEBUG    topotato:frr.py:213 ignoring target 'ospfclient/ospfclient'
DEBUG    topotato:frr.py:213 ignoring target 'pimd/test_igmpv3_join'
DEBUG    topotato:frr.py:213 ignoring target 'pceplib/pcep_pcc'
DEBUG    topotato:topolinux.py:91 executable unshare found: /usr/bin/unshare
DEBUG    topotato:topolinux.py:91 executable nsenter found: /usr/bin/nsenter
DEBUG    topotato:topolinux.py:91 executable tini found: /usr/bin/tini
DEBUG    topotato:topolinux.py:91 executable ip found: /usr/sbin/ip
Warning: daemon 'pim6d' not enabled in configure, skipping
================================================================================ test session starts ================================================================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.11.0, pluggy-0.13.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /root/basetato4, configfile: pytest.ini
collecting ... -------------------------------------------------------------------------------- live log collection --------------------------------------------------------------------------------
DEBUG    topotato:base.py:275 _topotato_makeitem(<Module test_bgp_orf.py>, 'TestBase', <class 'topotato.base.TestBase'>)
DEBUG    topotato:base.py:275 _topotato_makeitem(<Module test_bgp_orf.py>, 'TestBGPORF', <class 'test_bgp_orf.TestBGPORF'>)
DEBUG    topotato:base.py:275 _topotato_makeitem(<Instance ()>, 'bgp_converge_r1', <topotato.base.TopotatoWrapped object at 0x7f50af3d6760>)
DEBUG    topotato:base.py:275 _topotato_makeitem(<Instance ()>, 'bgp_converge_r2', <topotato.base.TopotatoWrapped object at 0x7f50af3d6610>)
DEBUG    topotato:base.py:275 _topotato_makeitem(<Instance ()>, 'bgp_orf_changed_r1', <topotato.base.TopotatoWrapped object at 0x7f50af3d6670>)
DEBUG    topotato:base.py:275 _topotato_makeitem(<Instance ()>, 'bgp_orf_changed_r2', <topotato.base.TopotatoWrapped object at 0x7f50af3d66d0>)
DEBUG    topotato:base.py:682 collect on: <TopotatoFunction bgp_converge_r1> test: <AssertVtysh #85:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json]>
DEBUG    topotato:base.py:682 collect on: <TopotatoFunction bgp_converge_r2> test: <AssertVtysh #106:r2/bgpd/vtysh[show bgp ipv4 unicast summary json]>
DEBUG    topotato:base.py:682 collect on: <TopotatoFunction bgp_orf_changed_r1> test: <AssertVtysh #120:r2/zebra/vtysh[;             enable;             configure terminal;             ip prefix-list r1 seq 10 permit 10.255.0.2/32;             ]>
DEBUG    topotato:base.py:682 collect on: <TopotatoFunction bgp_orf_changed_r1> test: <AssertVtysh #131:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json]>
DEBUG    topotato:base.py:682 collect on: <TopotatoFunction bgp_orf_changed_r2> test: <AssertVtysh #148:r2/bgpd/vtysh[show bgp ipv4 unicast summary json]>
collected 7 items                                                                                                                                                                   

test_bgp_orf.py::TestBGPORF::startup 
---------------------------------------------------------------------------------- live log setup -----------------------------------------------------------------------------------
DEBUG    topotato.topolinux:topolinux.py:327 <topotato.frr.FRRNetworkInstance object at 0x7f50af3e9820> tempdir created: /tmp/tmp80macpxd
DEBUG    topotato.topolinux:topolinux.py:113 <topotato.frr.FRRNetworkInstance object at 0x7f50af3e9820> temp-subdir for <SwitchyNS: 'switch-ns'> created: /tmp/tmp80macpxd/switch-ns
DEBUG    topotato.topolinux:topolinux.py:113 <topotato.frr.FRRNetworkInstance object at 0x7f50af3e9820> temp-subdir for <RouterNS: 'r1'> created: /tmp/tmp80macpxd/r1
DEBUG    topotato.topolinux:topolinux.py:113 <topotato.frr.FRRNetworkInstance object at 0x7f50af3e9820> temp-subdir for <RouterNS: 'r2'> created: /tmp/tmp80macpxd/r2
PASSED (3.92)                                                                                                                                                                 [ 14%]
test_bgp_orf.py::TestBGPORF::bgp_converge_r1:#85:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json] PASSED (1.90)                                [ 28%]
test_bgp_orf.py::TestBGPORF::bgp_converge_r2:#106:r2/bgpd/vtysh[show bgp ipv4 unicast summary json] PASSED (0.00)                                                             [ 42%]
test_bgp_orf.py::TestBGPORF::bgp_orf_changed_r1:#120:r2/zebra/vtysh[;             enable;             configure terminal;             ip prefix-list r1 seq 10 permit 10.255.0.2/32;             ] PASSED (0.01) [ 57%]
test_bgp_orf.py::TestBGPORF::bgp_orf_changed_r1:#131:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json] FAILED                                   [ 71%]

===================================================================================== FAILURES ======================================================================================
_______________________________________________ #131:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json] ________________________________________________

self = <test_bgp_orf.TestBGPORF object at 0x7f50af3d64c0>, r1 = <Router 1 "r1">, r2 = <Router 2 "r2">

    @topotatofunc
    def bgp_orf_changed_r1(self, r1, r2):

        expected = {"advertisedRoutes": {str(r1.lo_ip4[0]): {}, str(r2.lo_ip4[0]): {}}}

        yield from AssertVtysh.make(
            r2,
            "zebra",
            f"""
            enable
            configure terminal
            ip prefix-list r1 seq 10 permit {r2.lo_ip4[0]}
            """,
            compare="",
        )

>       yield from AssertVtysh.make(
            r1,
            "bgpd",
            f"show bgp ipv4 unicast neighbor {r2.iface_to('s1').ip4[0].ip} advertised-routes json",
            maxwait=5.0,
            compare=expected,
        )
E       topotato.exceptions.TopotatoCLICompareFail: expected key(s) ['10.255.0.2/32'] in json["advertisedRoutes"] (have ['10.255.0.1/32']):
E       --- Expected value
E       +++ Current value
E       @@ -2,2 +2,14 @@
E       -    "10.255.0.1/32": {},
E       -    "10.255.0.2/32": {}
E       +    "10.255.0.1/32": {
E       +        "addrPrefix": "10.255.0.1",
E       +        "appliedStatusSymbols": {
E       +            "*": true,
E       +            ">": true
E       +        },
E       +        "bgpOriginCode": "?",
E       +        "metric": 0,
E       +        "network": "10.255.0.1/32",
E       +        "nextHop": "0.0.0.0",
E       +        "path": "",
E       +        "prefixLen": 32,
E       +        "weight": 32768
E       +    }

/root/basetato4/test_bgp_orf.py:131: TopotatoCLICompareFail
============================================================================== short test summary info ==============================================================================
FAILED test_bgp_orf.py::TestBGPORF::bgp_orf_changed_r1:#131:r1/bgpd/vtysh[show bgp ipv4 unicast neighbor 10.101.0.2 advertised-routes json] - topotato.exceptions.TopotatoCLICompa...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================== 1 failed, 4 passed in 11.49s ============================================================================
eznix86 commented 1 year ago

Can I take over this ?

Chromico commented 1 year ago

@eznix86 Sure, go for it.

eqvinox commented 1 year ago

reworked & merged in ba36b4ffce1aab3677a9beefac4bc6e44c977156