BGP-LS ipv6 information - exabgp process is hanging

thomas955 commented 1 year ago

Bug Report

We are sorry that you are experiencing an issue with ExaBGP.

Before opening this issue could you please:

make sure the problem was not already reported to avoid duplicates.
- Checked
check if the problem is still present on the latest version (master)
- commit a33b3ce7a2e5082009a84e12e779e44096156ebf (HEAD -> 4.2, origin/4.2)
provide an full output of exabgp running with the "-d" option, including any stacktrace (please provide the full output - even if long)

Describe the bug Exabgp hangs / stalls if it gets IPv6 interface information distributed from is-is over address family bgp-ls.

To reproduce it I used a CiscoIOSXR and on Looppback 0 i added an ipv6 address like 2000::1/128. I will give you some shortened config parts out of the cisco config with the most important parts:

!
interface Loopback0
 ipv4 address 172.172.0.1 255.255.255.255
 ipv6 address 2000::1/128
 ipv6 enable
!
...
!
router isis 1
 is-type level-2-only
 net 10.0000.0000.0000.0001.00
 distribute link-state
...
 interface Loopback0
  passive
  point-to-point
  address-family ipv4 unicast
  !
  address-family ipv6 unicast
  !
 !
!
router bgp 2
 address-family ipv4 unicast
 !
 address-family link-state link-state
 !
 neighbor 10.11.13.2
  remote-as 2
  address-family ipv4 unicast
  !
  address-family link-state link-state
  !
 !
!

Now the straight forward exabgp config:

neighbor 10.11.13.1 {
        local-address 10.11.13.2;
        local-as 2;
        peer-as 2;
        family {
                   #ipv4 unicast;
                   bgp-ls bgp-ls;
        }
}

And finnaly the debug output from exabgp:

15:59:12 | 15724  | outgoing-1    | received complete TCP payload (  19) FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 0085 02
15:59:12 | 15724  | outgoing-1    | received complete TCP payload ( 114) 0000 006E 900E 0052 4004 4704 0A0B 0D01 0000 0200 4502 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0101 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0002 4001 0100 4002 0040 0504 0000 0064 801D 0704 4700 0300 000A
15:59:12 | 15724  | outgoing-1    | << message of type UPDATE
15:59:12 | 15724  | parser        | parsing UPDATE ( 114) 0000 006E 900E 0052 4004 4704 0A0B 0D01 0000 0200 4502 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0101 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0002 4001 0100 4002 0040 0504 0000 0064 801D 0704 4700 0300 000A
15:59:12 | 15724  | routes        | withdrawn NLRI none
15:59:12 | 15724  | parser        | attribute mp-reach-nlri      flag 0x90 type 0x0e len 0x52 payload 4004 4704 0A0B 0D01 0000 0200 4502 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0101 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0002
15:59:12 | 15724  | parser        | NLRI      bgp-ls bgp-ls      without path-information     payload 0002 0045 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 0101 0100 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 02
15:59:12 | 15724  | parser        | attribute origin             flag 0x40 type 0x01 len 0x01 payload 00
15:59:12 | 15724  | parser        | attribute as-path            flag 0x40 type 0x02 len 0x00
15:59:12 | 15724  | parser        | attribute local-preference   flag 0x40 type 0x05 len 0x04 payload 0000 0064
15:59:12 | 15724  | parser        | attribute bgp-ls             flag 0x80 type 0x1d len 0x07 payload 0447 0003 0000 0A
15:59:12 | 15724  | routes        | announced NLRI none
15:59:12 | 15724  | parser        | decoded UPDATE (   0) json { "exabgp": "4.0.1", "time": 1665669552.2212465, "host" : "thomas-Latitude-7490", "pid" : 15724, "ppid" : 3232, "counter": 1, "type": "update", "neighbor": { "address": { "local": "10.11.13.2", "peer": "10.11.13.1" }, "asn": { "local": 2, "peer": 2 } , "direction": "in", "message": { "update": { "attribute": { "origin": "igp", "local-preference": 100, "bgp-ls": { "igp-metric": 10 } }, "announce": { "bgp-ls bgp-ls": { "10.11.13.1": [ { "ls-nlri-type": "bgpls-link", "l3-routing-topology": 0, "protocol-id": 2, "local-node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000001" }, "remote-node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000002" }, "interface-address": {  }, "neighbor-address": {  } } ] } } } } } }
15:59:12 | 15724  | peer-1        | << UPDATE #1
15:59:12 | 15724  | peer-1        |    UPDATE #1 nlri  (   2) { "ls-nlri-type": "bgpls-link", "l3-routing-topology": 0, "protocol-id": 2, "local-node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000001" }, "remote-node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000002" }, "interface-address": {  }, "neighbor-address": {  } }
15:59:12 | 15724  | outgoing-1    | received complete TCP payload (  19) FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 0076 02
15:59:12 | 15724  | outgoing-1    | received complete TCP payload (  99) 0000 005F 900E 0034 4004 4704 0A0B 0D01 0000 0100 2702 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 4001 0100 4002 0040 0504 0000 0064 801D 1601 0700 0400 0000 0204 0200 0369 6F73 0403 0003 1000 00
15:59:12 | 15724  | outgoing-1    | << message of type UPDATE
15:59:12 | 15724  | parser        | parsing UPDATE (  99) 0000 005F 900E 0034 4004 4704 0A0B 0D01 0000 0100 2702 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 4001 0100 4002 0040 0504 0000 0064 801D 1601 0700 0400 0000 0204 0200 0369 6F73 0403 0003 1000 00
15:59:12 | 15724  | routes        | withdrawn NLRI none
15:59:12 | 15724  | parser        | attribute mp-reach-nlri      flag 0x90 type 0x0e len 0x34 payload 4004 4704 0A0B 0D01 0000 0100 2702 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001
15:59:12 | 15724  | parser        | NLRI      bgp-ls bgp-ls      without path-information     payload 0001 0027 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 01
15:59:12 | 15724  | parser        | attribute origin             flag 0x40 type 0x01 len 0x01 payload 00
15:59:12 | 15724  | parser        | attribute as-path            flag 0x40 type 0x02 len 0x00
15:59:12 | 15724  | parser        | attribute local-preference   flag 0x40 type 0x05 len 0x04 payload 0000 0064
15:59:12 | 15724  | parser        | attribute bgp-ls             flag 0x80 type 0x1d len 0x16 payload 0107 0004 0000 0002 0402 0003 696F 7304 0300 0310 0000
15:59:12 | 15724  | routes        | announced NLRI none
15:59:12 | 15724  | parser        | decoded UPDATE (   0) json { "exabgp": "4.0.1", "time": 1665669552.2226422, "host" : "thomas-Latitude-7490", "pid" : 15724, "ppid" : 3232, "counter": 2, "type": "update", "neighbor": { "address": { "local": "10.11.13.2", "peer": "10.11.13.1" }, "asn": { "local": 2, "peer": 2 } , "direction": "in", "message": { "update": { "attribute": { "origin": "igp", "local-preference": 100, "bgp-ls": { "attribute-not-implemented": "263", "node-name": "ios", "area-id": "100000" } }, "announce": { "bgp-ls bgp-ls": { "10.11.13.1": [ { "ls-nlri-type": "bgpls-node", "l3-routing-topology": 0, "protocol-id": 2, "node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000001" }, "nexthop": "10.11.13.1" } ] } } } } } }
15:59:12 | 15724  | peer-1        | << UPDATE #2
15:59:12 | 15724  | peer-1        |    UPDATE #2 nlri  (   2) { "ls-nlri-type": "bgpls-node", "l3-routing-topology": 0, "protocol-id": 2, "node-descriptors": { "autonomous-system": 2, "bgp-ls-identifier": "0", "router-id": "000000000001" }, "nexthop": "10.11.13.1" }
15:59:12 | 15724  | outgoing-1    | received complete TCP payload (  19) FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 00CE 02
15:59:12 | 15724  | outgoing-1    | received complete TCP payload ( 187) 0000 00B7 900E 0095 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 0140 0101 0040 0200 4005 0400 0000 6480 1D0D 0483 0004 0000 0000 0492 0001 20
15:59:12 | 15724  | outgoing-1    | << message of type UPDATE
15:59:12 | 15724  | parser        | parsing UPDATE ( 187) 0000 00B7 900E 0095 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 0140 0101 0040 0200 4005 0400 0000 6480 1D0D 0483 0004 0000 0000 0492 0001 20
15:59:12 | 15724  | routes        | withdrawn NLRI none
15:59:12 | 15724  | parser        | attribute mp-reach-nlri      flag 0x90 type 0x0e len 0x95 payload 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109
15:59:12 | 15724  | parser        | NLRI      bgp-ls bgp-ls      without path-information     payload 0004 0042 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 0101 0700 0200 0201 0900 1180 2022 0000 0000 0000 0000 0000 0000 0001 0004 0042 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 0101 0700 0200 0201 0900 1180 2000 0000 0000 0000 0000 0000 0000 0001

After this it hangs in some kind of endless loop. No access to cli etc anymore nor more logging.

Environment (please complete the following information):

Ubuntu 18.04
see above the git commit from branch 4.2

Thank you in advance.

thomas-mangin commented 1 year ago

The code was not handling unknown TLV type and looping forever, it is now on master a bit less worse, it will kill the session.

./sbin/exabgp decode -d "0000 00B7 900E 0095 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 0140 0101 0040 0200 4005 0400 0000 6480 1D0D 0483 0004 0000 0000 0492 0001 20
"
parser          parsing UPDATE ( 187) 0000 00B7 900E 0095 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 0140 0101 0040 0200 4005 0400 0000 6480 1D0D 0483 0004 0000 0000 0492 0001 20
parser          attribute mp-reach-nlri      flag 0x90 type 0x0e len 0x95 payload 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 01
parser          NLRI      bgp-ls bgp-ls      without path-information     payload 0004 0042 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 0101 0700 0200 0201 0900 1180 2022 0000 0000 0000 0000 0000 0000 0001 0004 0042 0200 0000 0000 0000 0001 0000 1A02 0000 0400 0000 0202 0100 0400 0000 0002 0300 0600 0000 0000 0101 0700 0200 0201 0900 1180 2000 0000 0000 0000 0000 0000 0000 0001
invalid payload

thomas-mangin commented 1 year ago

Now reporting that we could not parse part of the TLV instead.

❯ ./sbin/exabgp decode "0000 00B7 900E 0095 4004 4704 0A0B 0D01 0000 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 2200 0000 0000 0000 0000 0000 0000 0100 0400 4202 0000 0000 0000 0000 0100 001A 0200 0004 0000 0002 0201 0004 0000 0000 0203 0006 0000 0000 0001 0107 0002 0002 0109 0011 8020 0000 0000 0000 0000 0000 0000 0000 0140 0101 0040 0200 4005 0400 0000 6480 1D0D 0483 0004 0000 0000 0492 0001 20
"
              unknown prefix v6 TLV 263
              unknown prefix v6 TLV 263
{ "exabgp": "5.0.0", "time": 1668781676.4027362, "host" : "MacBook-Pro-2.local", "pid" : 72678, "ppid" : 65389, "counter": 1, "type": "update", "neighbor": {     "address": { "local": "127.0.0.1", "peer": "127.0.0.1" },     "asn": { "local": 65533, "peer": 65533 }     , "direction": "in", "message": { "update": { "attribute": { "origin": "igp", "local-preference": 100, "bgp-ls": { "prefix-metric": 0, "sr-prefix-attribute-flags": {"X": 0, "R": 0, "N": 1, "RSV": 0} } }, "announce": { "bgp-ls bgp-ls": { "10.11.13.1": [ { "ls-nlri-type": "bgpls-prefix-v6", "l3-routing-topology": 0, "protocol-id": 2, "node-descriptors": [ { "autonomous-system": 2 }, { "bgp-ls-identifier": "0" }, { "router-id": "000000000001" } ], "ip-reachability-tlv": "2022::1", "ip-reach-prefix": "2022::1/128", "nexthop": "10.11.13.1" }, { "ls-nlri-type": "bgpls-prefix-v6", "l3-routing-topology": 0, "protocol-id": 2, "node-descriptors": [ { "autonomous-system": 2 }, { "bgp-ls-identifier": "0" }, { "router-id": "000000000001" } ], "ip-reachability-tlv": "2000::1", "ip-reach-prefix": "2000::1/128", "nexthop": "10.11.13.1" } ] } } } } } }

As I did not author that code and have no time right now to look at the RFC to figure out what is happening, I think it is a good compromise.

A Multi-Topology Identifier is being passed to some class which did not expect it. If it valid, the class should be extended to allow it.

thomas-mangin commented 1 year ago

https://www.rfc-editor.org/rfc/rfc7752.html#section-3.3.3 does not have MTID and as I was not the primary author of the BGPLS code, I can not see why this should be valid. If this was changed by a later RFC, I would need some pointers please.

thomas-mangin commented 1 year ago

I am closing but feel free to re-open if you can point me in the right direction.

Exa-Networks / exabgp

BGP-LS ipv6 information - exabgp process is hanging #1125