Flowspec redirect to IP not working with XR 5.3.1 ?

[mmarchand]

Hi,

I am trying to setup a redirect-to-ip rule from exabgp (3.4.11) to IOS XR 5.3.1 but XR does not seem to receive the appropriate extended communities and so does not redirect the traffic.

ExaBGP log says :

reactor       | Flow added to neighbor 1.2.3.4 local-ip 5.6.7.8 local-as 65500 peer-as 65500 router-id 5.6.7.8 family-allowed in-open : flow destination a.b.c.d/32 next-hop e.f.g.h extended-community redirect-to-nexthop 
which looks fine to me

but XR sees:

#show bgp ipv4 flowspec Dest:a.b.c.d/32/48 detail
Fri Jul 17 12:08:29.515 UTC
BGP routing table entry for Dest:a.b.c.d/32/48
NLRI in Hex: 0120c39ab2a3/48
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                106         106
    Flags: 0x04001001+0x00000000;
Last Modified: Jul 17 12:08:24.157 for 00:00:05
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 1
  Flags: 0x4000000001060005, import: 0x20
  Not advertised to any peer
  Local
    e.f.g.h (metric 100) from 5.6.7.8 (5.6.7.8)
      Origin IGP, localpref 100, valid, confed-internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 106

if I set the same rule locally on XR (with class-map/policy-map), I get almost the same output but with these communities added (and with this it actually redirects traffic) :

      "Extended community: FLOWSPEC Redirect-IP:0"

I use the following flow syntax in ExaBGP :

"neighbor %s announce flow route { match { destination %s; } then { redirect %s; } }\n"

also tried with :

"neighbor %s announce flow route { next-hop e.f.g.h; match { destination %s; } then { redirect-to-nexthop; } }\n"

am I forgetting something or are the extended communities attributes not set correctly in exabgp ? (also tried adding some extended-communities [ xxx:yyy ] and those where properly advertised to XR)

thanks Mik

[mmarchand]

also replacing redirect/redirect-to-nexthop with a discard action, I correctly get the communities in XR (and traffic is discarded) : Extended community: FLOWSPEC Traffic-rate:0,0

[thomas-mangin]

Hi @mmarchand

Sorry for this. Could you please send me a raw BGP dump ( using the -d option of exabgp ) of what Cisco generates for this flow so I can :

• check my parser to make sure I correctly understood the encoding / decoding of the community
• can check how my update generation differs

It would help me a lot fixing this issue. Thanks.

[mmarchand]

first time playing with -d , tell me if it's wrong/not enough ;) this should contain a new announce then a withdrawal of a redirect-to-ip from XR :

Fri, 17 Jul 2015 14:47:40 | DEBUG    | 64356  | wire          | session 1 outgoing 5.6.7.8 / 1.2.3.4     RECEIVED  (47) FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 0044 02
Fri, 17 Jul 2015 14:47:40 | DEBUG    | 64356  | wire          | session 1 outgoing 5.6.7.8 / 1.2.3.4     RECEIVED  (122) 0000 002D 900E 0010 0001 8504 3ED2 AFDD 0006 0120 C39A B2A3 4001 0100 4002 0040 0504 0000 0064 C010 0808 0000 0000 0000 00
Fri, 17 Jul 2015 14:47:40 | INFO     | 64356  | routes        | peer 1.2.3.4 ASN 65500   << UPDATE (4) (0)
Fri, 17 Jul 2015 14:47:46 | DEBUG    | 64356  | wire          | session 1 outgoing 5.6.7.8 / 1.2.3.4     RECEIVED  (47) FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 0025 02
Fri, 17 Jul 2015 14:47:46 | DEBUG    | 64356  | wire          | session 1 outgoing 5.6.7.8 / 1.2.3.4     RECEIVED  (44) 0000 000E 900F 000A 0001 8506 0120 C39A B2A3

thanks Mik

[thomas-mangin]

./sbin/exabgp qa/conf/flow.conf --decode "0000 002D 900E 0010 0001 8504 3ED2 AFDD 0006 0120 C39A B2A3 4001 0100 4002 0040 0504 0000 0064 C010 0808 0000 0000 0000 00"

[...]
Fri, 17 Jul 2015 14:04:18 | INFO     | 84983  | parser        | decoded update 1 flow destination 195.154.178.163/32 next-hop 62.210.175.221 origin igp local-preference 100 extended-community 0x0800000000000000
Fri, 17 Jul 2015 14:04:18 | INFO     | 84983  | parser        | update json { "exabgp": "3.4.0", "time": 1437138258, "host" : "mangin.local", "pid" : "84983", "ppid" : "82378", "counter": 1, "type": "update", "neighbor": { "ip": "127.0.0.1", "address": { "local": "127.0.0.1", "peer": "127.0.0.1"}, "asn": { "local": "1", "peer": "1"}, "message": { "update": { "attribute": { "origin": "igp", "local-preference": 100, "extended-community": [ 576460752303423488 ] }, "announce": { "ipv4 flow": { "62.210.175.221": { "flow-11": { "destination": [ "195.154.178.163/32" ], "next-hop": "62.210.175.221", "string": "flow destination 195.154.178.163/32 next-hop 62.210.175.221" } } } } } }} }

[thomas-mangin]

Ok - it would be better to add a community for next-hop and I have 5 lines patch for that, so I will improve the format in 4.0.0 but in your case please try:

"neighbor %s announce flow route { next-hop %s; match { destination %s; } then { redirect-to-nexthop; } }\n"

redirect is for RFC5575 redirection and you want draft-simpson-idr-flowspec-redirect-02, if this does not generate the right update. I will have to investigate out why.

[thomas-mangin]

https://github.com/Exa-Networks/exabgp/blob/3.4/lib/exabgp/configuration/ancient.py#L200

[mmarchand]

that would be :

"neighbor %s announce flow route { next-hop e.f.g.h; match { destination %s; } then { redirect-to-nexthop; } }\n"

but I already tried this syntax and this does not set the extended communities either :(

[thomas-mangin]

That the part where I run away ... or start looking into why !

[mmarchand]

the decoder says exabgp sends for this flow : "extended-community": [ 9223372036854775808 ] to compare to XR's "extended-community": [ 576460752303423488 ] probably (not sure how to split the various parts of it :)

[nfevrier]

Hey Thomas, Mickael, FYI, the XR implementation is based on: https://www.ietf.org/id/draft-ietf-idr-flowspec-redirect-rt-bis-04.txt So, the expected extended community for redirect-to-IPv4 is 0x8108. Cheers, Nicolas.

[thomas-mangin]

@nfevrier this is surprising as it is not what the XR is sending back to ExaBGP.

C0 - Flag
10 - Extended Community
08 - Size
08 0000 0000 0000 00 - Content

I can change the code to make it work but then I am breaking some other tests. So this is not a 5 minutes fix as I hoped (now counting in hours).

Taking the 3.4 stable tree branch and applying this patch may work (it may/will also break other things you may not care about).

diff --git a/lib/exabgp/bgp/message/update/attribute/community/extended/traffic.py b/lib/exabgp/bgp/message/update/attribute/community/extended/traffic.py
index 04614f7..3f55fb3 100644
--- a/lib/exabgp/bgp/message/update/attribute/community/extended/traffic.py
+++ b/lib/exabgp/bgp/message/update/attribute/community/extended/traffic.py
@@ -156,7 +156,7 @@ class TrafficMark (ExtendedCommunity):
 # XXX: FIXME: I guess this should be a subclass of NextHop or IP ..

 class TrafficNextHop (ExtendedCommunity):
-       COMMUNITY_TYPE = 0x80
+       COMMUNITY_TYPE = 0x08
        COMMUNITY_SUBTYPE = 0x00

        __slots__ = ['copy']

This should generate what XR wants but it looks wrong to me and if it works can not be supported as it.

[nfevrier]

Gimme a couple of days to confirm with engineering and I'll get back to you. Thanks, N.

[mmarchand]

just tested the patch, you also need to change the 0x80 to 0x08 just below in the code and then it does indeed work ;)

[mmarchand]

sorry in branch 3.4 "master" it would be okay indeed, it's been refactored 3.4.11 needs the second tweak

[thomas-mangin]

@nfevrier thank you, I am surprised by the output, it looks wrong to me Reading http://www.iana.org/assignments/bgp-extended-communities/bgp-extended-communities.xhtml#transitive I would have understood 0x8008 0x00000000 but not 0x0800 0x000000000 as we see

[thomas-mangin]

@mmarchand I was due to release 3.4.12 which has other fixes, so I will do this week-end.

[thomas-mangin]

@mmarchand this is even weirder .. so the code needs 0x0808 0x000000 as community ?

[thomas-mangin]

https://www.ietf.org/id/draft-ietf-idr-flowspec-redirect-rt-bis-04.txt says :

   +--------+--------------------+-------------------------------------+
   | type   | extended community | encoding                            |
   +--------+--------------------+-------------------------------------+
   | 0x8008 | redirect AS-2byte  | 2-octet AS, 4-octet Value           |
   | 0x8108 | redirect IPv4      | 4-octet IPv4 Address, 2-octet Value |
   | 0x8208 | redirect AS-4byte  | 4-octet AS, 2-octet Value           |
   +--------+--------------------+-------------------------------------+

So I could accept having an issue with encoding and needing 0x8108 if https://tools.ietf.org/html/draft-ietf-idr-flowspec-redirect-ip is what I should have used.

[mmarchand]

just to be clear here is the function updated:

class TrafficNextHop (ExtendedCommunity):
        COMMUNITY_TYPE = 0x08
        COMMUNITY_SUBTYPE = 0x00

        __slots__ = ['copy']

        def __init__ (self, copy, community=None):
                self.copy = copy
                ExtendedCommunity.__init__(
                        self,
                        community if community is not None else pack(
                                "!BBLH",
                                0x08,0x00,
                                0,1 if copy else 0
                        )
                )

        def __str__ (self):
                return "copy-to-nexthop" if self.copy else "redirect-to-nexthop"

        @staticmethod
        def unpack (data):
                bit, = unpack('!B',data[7])
                return TrafficNextHop(bool(bit & 0x01),data[:8])

without the second 0x80->0x08 change , XR sees :

RP/0/RP0/CPU0:bb1.dc3#show bgp ipv4 flowspec  Dest:195.154.178.163/32/48
Fri Jul 17 15:31:44.024 UTC
BGP routing table entry for Dest:195.154.178.163/32/48
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                166         166
Last Modified: Jul 17 15:31:33.738 for 00:00:10
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 1
  Not advertised to any peer
  Local
    62.210.175.221 (metric 100) from a.b.c.d (a.b.c.d)
      Origin IGP, localpref 100, valid, confed-internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 166

and with the second change :

RP/0/RP0/CPU0:bb1.dc3#show bgp ipv4 flowspec  Dest:195.154.178.163/32/48
Fri Jul 17 15:33:39.374 UTC
BGP routing table entry for Dest:195.154.178.163/32/48
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                175         175
Last Modified: Jul 17 15:33:37.734 for 00:00:01
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 1
  Not advertised to any peer
  Local
    62.210.175.221 (metric 100) from a.b.c.d (a.b.c.d)
      Origin IGP, localpref 100, valid, confed-internal, best, group-best
      Received Path ID 0, Local Path ID 1, version 175
      Extended community: FLOWSPEC Redirect-IP:0

[thomas-mangin]

@mmarchand Thank you for the clarification. @tmmorin refactored this code to prevent this duplication already, so I missed it when writing the patch :p

[thomas-mangin]

@tmmorin @mmarchand @nfevrier

Having quickly looked at the difference between the drafts, the -rt way of encoding does not require to use the next-hop field. Having to use the next hop filed is a pain to implement but does not limit redirection to IPv4 addresses ( unless a way to encode the IPv6 using multiple community is proposed but that would be ugly )

My issue is that it seems both draft are incompatible .. is someone going to address this at Prague, or was it already done as the -rt version got the IANA allocation ?

[nfevrier]

Hi Thomas, Michael, after talking to engineering, I realise I'm mixing up things and created a confusion. 0x8108 is not for redirect-to-IP but it's a redirect-to-RT with the RT expressed in the IPv4 form: In our CLI format, it's the third of this list:

RP/0/0/CPU0:XRv1-demo(config-pmap-c)#redirect nexthop route-target ?

<1-65535>: 2-byte AS number <65536-4294967295>: 4-byte AS number in asplain format A.B.C.D: IPv4 address RP/0/0/CPU0:XRv1-demo(config-pmap-c)#

So it's not https://www.ietf.org/id/draft-ietf-idr-flowspec-redirect-rt-bis-04.txt that should be used here. 0x08 is indeed currently used in our implementation. Sorry again for the confusion. N.

[nfevrier]

And to conclude on this point, IOS XR implementation of the redirect-to-IP (and very much likely IOS XE since it reuses a lot of XR code for BGP FS) is based on this draft: https://tools.ietf.org/html/draft-simpson-idr-flowspec-redirect-02

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     0x08      |     0x00      |     Reserved                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     (Set to zero and          |
|                                     ignored on receipt)     |C|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note that we didn't implemented https://tools.ietf.org/html/draft-ietf-idr-flowspec-redirect-ip-02 yet. Thanks, N.

[xiaopeng163]

The ext-community code for redirect to RT is 0x8008 and redirect to nexthop is 0x8000.

I think Cisco implemented both. Like redirect to nexthop

RP/0/0/CPU0:XR5#sh flowspec ipv4 detail 
Wed Jul 22 03:45:04.210 UTC

AFI: IPv4
 Flow           :Dest:192.85.2.3/32,Source:192.85.1.3/32
  Actions       :Traffic-rate: 50000000 bps DSCP: cs3 Nexthop: 192.85.3.3  (policy.1.fs)
RP/0/0/CPU0:XR5#

[nfevrier]

@xiaopeng163 Hi Peng Xiao, Here is the list of the extcomm Cisco implemented for flowspec (IOS XR images from 520 to 531) Traffic rate: 0x8006 Traffic action: 0x8007 Redirect RT ASN2B: 0x8008 Redirect RT V4 addr: 0x8108 Redirect RT ASN4B: 0x8208 Traffic Mark: 0x8009 Redirect IP NH: 0x0800 <-- Note: not 0x8000 Kind regards, N.

[xiaopeng163]

@nfevrier thanks for that

[thomas-mangin]

AFAICR this is now fully fixed and I can now close the issue.