alice-lg / birdwatcher

A JSON API for BIRD
BSD 3-Clause "New" or "Revised" License
66 stars 41 forks source link

panic: interface conversion: interface {} is nil, not string #35

Open tweippert opened 3 years ago

tweippert commented 3 years ago

birdwatcher (v4) is bailing out with the following messages:

[ ... ]
172.18.20.3 - - [12/Mar/2021:22:50:31 +0100] "GET //routes/table/master HTTP/1.1" 301 55
panic: interface conversion: interface {} is nil, not string

goroutine 31 [running]:
github.com/alice-lg/birdwatcher/bird.parseRouteLines(0x9080400, 0xf, 0x10, 0x4, 0x92d21c0)
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:335 +0xe09
github.com/alice-lg/birdwatcher/bird.workerForRouteBlockParsing(0x92d2180, 0x92d21c0, 0x930c330)
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:313 +0x87
created by github.com/alice-lg/birdwatcher/bird.startRouteWorkers.func1
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:271 +0x4b

the v6 version is running without issues.

goochjj commented 3 years ago

Happens to me on ipv4, with lines:

192.0.2.1/32 blackhole [static1 2021-05-02 01:09:54] * (200) Type: static univ unicast [SK12_RR1V4 2021-05-07 21:19:34] (100) [I] via 169.254.1.253 on ext0 Type: BGP univ BGP.origin: IGP BGP.as_path: BGP.next_hop: 169.254.0.253 BGP.local_pref: 100

Also happens on IPv6 with my 100::/64 prefix. I've added 192.0.2.1/32 and 100::/64 as static routes in bird, with the blackhole destination My route reflector is also advertising that route

I think birdwatcher is tripping over seeing the blackhole first (with no interface) and then seeing the next route which HAS an interface.

This

+       defer func() {
+               if err := recover(); err!= nil {
+                       log.Println("PANIC ", err);
+                       log.Println("lines ", lines);
+                       log.Println("route ", route);
+               }
+       }();

Was added to parseRouteLines on or around line 317 to diagnose the issue PANIC interface conversion: interface {} is nil, not string lines (above) route map[type:[static univ]]

goochjj commented 3 years ago

Trapping the panic allows birdwatcher to continue - but I also filtered out the blackhole advertisements on the RR and that fixed the issue as well.

goochjj commented 3 years ago

This fixes my issue

@@ -86,7 +87,7 @@ func init() {
        regex.routes.largeCommunity = regexp.MustCompile(`^\((\d+),\s*(\d+),\s*(\d+)\)`)
        regex.routes.extendedCommunity = regexp.MustCompile(`^\(([^,]+),\s*([^,]+),\s*([^,]+)\)`)
        regex.routes.origin = regexp.MustCompile(`\([^\(]*\)\s*`)
-       regex.routes.prefixBird2 = regexp.MustCompile(`^(` + re_prefix + `)?\s+unicast\s+\[([\w\.:]+)\s+([0-9\-\:\s]+)(
?:\s+from\s+(` + re_prefix + `))?\]\s+(?:(\*)\s+)?\((\d+)(?:\/\d+)?(?:\/[^\)]*)?\).*$`)
+       regex.routes.prefixBird2 = regexp.MustCompile(`^(` + re_prefix + `)?\s+(?:unicast|blackhole)\s+\[([\w\.:]+)\s+(
[0-9\-\:\s]+)(?:\s+from\s+(` + re_prefix + `))?\]\s+(?:(\*)\s+)?\((\d+)(?:\/\d+)?(?:\/[^\)]*)?\).*$`)
        regex.routes.gatewayBird2 = regexp.MustCompile(`^\s+via\s+(` + re_ip + `)\s+on\s+(` + re_ifname + `)\s*$`)
        regex.routes.interfaceBird2 = regexp.MustCompile(`^\s+dev\s+(` + re_ifname + `)\s*$`)
 }
goochjj commented 3 years ago

@tweippert Guessing birdc show route all |grep / |grep -v unicast will expose the route that's causing the problem.

tweippert commented 3 years ago

@tweippert Guessing birdc show route all |grep / |grep -v unicast will expose the route that's causing the problem.

With this i see my RPKI Routes, i currently had no blackhole or similar routes on this RR:

rr01:~# birdc show route all |grep / |grep -v unicast
172.18.0.0/24-32 AS65088  [RPKI01 2021-03-27 09:55:30] * (100)
172.18.0.0/24-32 AS65087  [RPKI01 2021-03-27 09:55:30] * (100)
....

Will look on your fix if it also works on my issue, hopefully the next few days ...

goochjj commented 3 years ago

Interesting... Is there a reason you have your RPKI routes in your master table? Mine are in separate tables since birdwatcher+alice don't really need to show those. (just the tags set by the BGP policy)

i.e.

roa4 table t_roa;
protocol rpki {
  roa4 { table t_roa; };

  remote 127.0.0.1 port 8282;
  retry keep 90;
  refresh keep 900;
  expire keep 172800;
}

function is_v4_rpki_invalid () {
  return roa_check(t_roa, net, bgp_path.last_nonaggregated) = ROA_INVALID;
}

roa6 table t_roa6;
protocol rpki {
  roa6 { table t_roa6; };

  remote 127.0.0.1 port 8282;
  retry keep 90;
  refresh keep 900;
  expire keep 172800;
}

function is_v6_rpki_invalid () {
  return roa_check(t_roa6, net, bgp_path.last_nonaggregated) = ROA_INVALID;
}
tweippert commented 3 years ago

I haven't. They are in roa tables, too. But a 'show route' shows me all tables:

Table r4:
172.18.0.0/24-32 AS65088  [RPKI01 2021-03-27 09:55:30] * (100)
172.18.0.0/24-32 AS65087  [RPKI01 2021-03-27 09:55:30] * (100)
...

Table r6:
2003:a:b67:f29::/64-64 AS65081  [RPKI01 2021-03-27 09:55:30] * (100)
2003:a:b67:f24::/64-64 AS65088  [RPKI01 2021-03-27 09:55:30] * (100)
...

My config looks similar:

roa4 table r4;
roa6 table r6;

protocol rpki RPKI01 {
        roa4 { table r4; };
        roa6 { table r6; };
        remote "rpki01.weiti.home" port 5323;
        retry keep 5;
        refresh keep 30;
        expire 600;
}
rr01:~# birdc show protocol all RPKI01 
BIRD 2.0.8 ready.
Name       Proto      Table      State  Since         Info
RPKI01     RPKI       ---        up     2021-05-09 15:11:53  Established
  Cache server:     rpki01.weiti.home
  Status:           Established
  Transport:        Unprotected over TCP
  Protocol version: 1
  Session ID:       27026
  Serial number:    0
  Last update:      before 24.169 s
  Refresh timer   : 5.830/30
  Retry timer     : ---
  Expire timer    : 7175.830/7200
  Channel roa4
    State:          UP
    Table:          r4
    Preference:     100
    Input filter:   ACCEPT
    Output filter:  REJECT
    Routes:         20 imported, 0 exported, 20 preferred
    Route change stats:     received   rejected   filtered    ignored   accepted
      Import updates:             20          0          0          0         20
      Import withdraws:            0          0        ---          0          0
      Export updates:              0          0          0        ---          0
      Export withdraws:            0        ---        ---        ---          0
  Channel roa6
    State:          UP
    Table:          r6
    Preference:     100
    Input filter:   ACCEPT
    Output filter:  REJECT
    Routes:         7 imported, 0 exported, 7 preferred
    Route change stats:     received   rejected   filtered    ignored   accepted
      Import updates:              7          0          0          0          7
      Import withdraws:            0          0        ---          0          0
      Export updates:              0          0          0        ---          0
      Export withdraws:            0        ---        ---        ---          0

This is with 2.08 on Alpine:

rr01:~# birdc
BIRD 2.0.8 ready.
goochjj commented 3 years ago

I have my alice-lg set to explicitly list table master and master6.... so maybe that's why I'm not tripping over it.

[source.bird4] name = BGPMon BIRD4 [source.bird4.birdwatcher] api = http://127.0.0.1:29184 show_last_reboot = true

type = multi_table

type = single_table

main_table = master4

insecure=true servertime = 2006-01-02T15:04:05.999999999Z07:00 servertime_short = 2006-01-02 15:04:05 servertime_ext = 2006-01-02 15:04:05

[source.bird6] name = BGPMon BIRD6 [source.bird6.birdwatcher] api = http://127.0.0.1:29184 show_last_reboot = true

type = multi_table

type = single_table main_table = master6 insecure=true servertime = 2006-01-02T15:04:05.999999999Z07:00 servertime_short = 2006-01-02 15:04:05 servertime_ext = 2006-01-02 15:04:05

tweippert commented 3 years ago

Yes, this seems to "fix" the panic ... will keep an eye on it the next hours/days.

No, it won't fix my issue, after few minutes the crash happens again ...

172.18.20.3 - - [11/May/2021:07:52:20 +0200] "GET //routes/table/master HTTP/1.1" 301 55
panic: interface conversion: interface {} is nil, not string

goroutine 32 [running]:
github.com/alice-lg/birdwatcher/bird.parseRouteLines(0x9480800, 0x10, 0x10, 0x4, 0x96d2280)
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:335 +0xe09
github.com/alice-lg/birdwatcher/bird.workerForRouteBlockParsing(0x96d2240, 0x96d2280, 0x970a1e0)
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:313 +0x87
created by github.com/alice-lg/birdwatcher/bird.startRouteWorkers.func1
        /home/weiti/go/src/github.com/alice-lg/birdwatcher/bird/parser.go:271 +0x4b
annikahannig commented 2 years ago

Merged the PR, however not sure if the problem is fixed.