kernel routes that match a zebra route get deleted from Tomahawk

I believe this is the same issue we're seeing in the k8's cluster when ospf adds a route, gobgp adds the same route but directly via the kernel, zebra see the kernel route and pulls the route it added.

Add a static route via quagga/zebra:

root@invader20:/etc/quagga# vtysh

Hello, this is Quagga (version 0.99.23.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

invader20# conf t
invader20(config)# ip route 1.1.1.0/24 10.100.0.11
invader20(config)# end
invader20# wr
Building Configuration...
Configuration saved to /etc/quagga/zebra.conf
[OK]
invader20# exit

Verify the route exists in kernel and came from proto zebra:

root@invader20:/etc/quagga# ip ro
default via 192.168.101.2 dev eth0 
1.1.1.0/24 via 10.100.0.11 dev eth-16-1  proto zebra

Verify the route exists in Tomahawk:

root@invader20:/etc/quagga# sudo goes vnet show ip fib
 Table                   Destination                               Adjacency
     default                    1.1.1.0/24      11: rewrite eth-16-1 IP4: 6c:ec:5a:07:cc:a0 -> 02:46:8a:00:05:c6 11-11, 1 x 10
     default                 10.100.0.0/24       3: glean eth-16-1
     default                10.100.0.11/32      10: rewrite eth-16-1 IP4: 6c:ec:5a:07:cc:a0 -> 02:46:8a:00:05:c6

Now add the same route using "ip" command:

root@invader20:/etc/quagga# ip ro ap 1.1.1.0/24 via 10.100.0.11

Verify route is in kernel (no longer owned by proto zebra):

root@invader20:/etc/quagga# ip ro
default via 192.168.101.2 dev eth0 
1.1.1.0/24 via 10.100.0.11 dev eth-16-1 
10.100.0.0/24 dev eth-16-1  proto kernel  scope link  src 10.100.0.20

Check Tomahawk fib (gone from fib):

root@invader20:/etc/quagga# sudo goes vnet show ip fib
 Table                   Destination                               Adjacency
     default                  10.32.0.0/24       5: glean eth-12-1
     default                 10.32.0.20/32       6: local eth-12-1
     default                 10.100.0.0/24       3: glean eth-16-1
     default                10.100.0.11/32      10: rewrite eth-16-1 IP4: 6c:ec:5a:07:cc:a0 -> 02:46:8a:00:05:c6
     default                10.100.0.20/32       4: local eth-16-1
                                              tx pipe packets                1
                                              tx pipe bytes                 98
     default                   30.0.0.0/31       9: glean eth-31-1
     default                   30.0.0.0/32       8: local eth-30-1
     default                   30.0.0.1/32       7: local eth-31-1

Using "ip monitor route" to see the netlink messages we see:

stig@invader20:~$ sudo ip monitor route all-nsid
[nsid current]1.1.1.0/24 via 10.100.0.11 dev eth-16-1  proto zebra 
[nsid current]1.1.1.0/24 via 10.100.0.11 dev eth-16-1 
[nsid current]Deleted 1.1.1.0/24 via 10.100.0.11 dev eth-16-1  proto zebra

1) added by zebra 2) added by kernel 3) zebra pulls it's original route

platinasystems / go

kernel routes that match a zebra route get deleted from Tomahawk #130