They all fail with the same zebra log errors:
zebra standard output:
2018/03/11 05:12:08 ZEBRA: VRF_GET: (NULL)(1)
2018/03/11 05:12:53 ZEBRA: release_daemon_chunks: Released 0 label chunks
2018/03/11 05:12:53 ZEBRA: client 18 disconnected. 0 system routes removed from the rib
2018/03/11 05:12:53 ZEBRA: release_daemon_chunks: Released 0 label chunks
2018/03/11 05:12:53 ZEBRA: client 17 disconnected. 0 system routes removed from the rib
2018/03/11 05:12:53 ZEBRA: Terminating on signal
2018/03/11 05:12:53 ZEBRA: vrf_terminate: Shutting down vrf subsystem
2018/03/11 05:12:53 ZEBRA: VRF 1 is to be deleted.
2018/03/11 05:12:53 ZEBRA: VRF 1 is to be disabled.
2018/03/11 05:12:53 ZEBRA: VRF 4294967295 is to be deleted.
ZEBRA: Received signal 11 at 1520770373 (si_addr 0x4, PC 0xb7f0af66); aborting...
Program counter: /usr/lib/libfrr.so.0(hash_clean+0x26)[0xb7f0af66]
Backtrace for 15 stack frames:
/usr/lib/libfrr.so.0(zlog_backtrace_sigsafe+0x48)[0xb7f13e58]
/usr/lib/libfrr.so.0(zlog_signal+0x340)[0xb7f14690]
/usr/lib/libfrr.so.0(+0x46ded)[0xb7f29ded]
[0xb7f8bcfc]
/usr/lib/libfrr.so.0(hash_clean+0x26)[0xb7f0af66]
/usr/lib/frr/zebra(zebra_ns_disable+0x2f)[0x4cb4cf]
/usr/lib/frr/zebra(zebra_ns_disabled+0x49)[0x4cb5f9]
/usr/lib/libfrr.so.0(ns_walk_func+0x37)[0xb7f18047]
/usr/lib/frr/zebra(+0x1aa97)[0x4b9a97]
/usr/lib/libfrr.so.0(quagga_sigevent_process+0x50)[0xb7f29ea0]
/usr/lib/libfrr.so.0(thread_fetch+0x608)[0xb7f36348]
/usr/lib/libfrr.so.0(frr_run+0xb6)[0xb7f12726]
/usr/lib/frr/zebra(main+0x318)[0x4ac5f8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb7d13637]
/usr/lib/frr/zebra(+0xdc45)[0x4acc45]
no thread information available
It was noticed recently that the netns tests are failing too often. After some investigation it seems that the bug lies in i386 VRF with netns.
Here are some samples of failing tests (i386):
They all fail with the same zebra log errors: zebra standard output:
2018/03/11 05:12:08 ZEBRA: VRF_GET: (NULL)(1)
2018/03/11 05:12:53 ZEBRA: release_daemon_chunks: Released 0 label chunks 2018/03/11 05:12:53 ZEBRA: client 18 disconnected. 0 system routes removed from the rib 2018/03/11 05:12:53 ZEBRA: release_daemon_chunks: Released 0 label chunks 2018/03/11 05:12:53 ZEBRA: client 17 disconnected. 0 system routes removed from the rib 2018/03/11 05:12:53 ZEBRA: Terminating on signal 2018/03/11 05:12:53 ZEBRA: vrf_terminate: Shutting down vrf subsystem 2018/03/11 05:12:53 ZEBRA: VRF 1 is to be deleted. 2018/03/11 05:12:53 ZEBRA: VRF 1 is to be disabled. 2018/03/11 05:12:53 ZEBRA: VRF 4294967295 is to be deleted. ZEBRA: Received signal 11 at 1520770373 (si_addr 0x4, PC 0xb7f0af66); aborting... Program counter: /usr/lib/libfrr.so.0(hash_clean+0x26)[0xb7f0af66] Backtrace for 15 stack frames: /usr/lib/libfrr.so.0(zlog_backtrace_sigsafe+0x48)[0xb7f13e58] /usr/lib/libfrr.so.0(zlog_signal+0x340)[0xb7f14690] /usr/lib/libfrr.so.0(+0x46ded)[0xb7f29ded] [0xb7f8bcfc] /usr/lib/libfrr.so.0(hash_clean+0x26)[0xb7f0af66] /usr/lib/frr/zebra(zebra_ns_disable+0x2f)[0x4cb4cf] /usr/lib/frr/zebra(zebra_ns_disabled+0x49)[0x4cb5f9] /usr/lib/libfrr.so.0(ns_walk_func+0x37)[0xb7f18047] /usr/lib/frr/zebra(+0x1aa97)[0x4b9a97] /usr/lib/libfrr.so.0(quagga_sigevent_process+0x50)[0xb7f29ea0] /usr/lib/libfrr.so.0(thread_fetch+0x608)[0xb7f36348] /usr/lib/libfrr.so.0(frr_run+0xb6)[0xb7f12726] /usr/lib/frr/zebra(main+0x318)[0x4ac5f8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb7d13637] /usr/lib/frr/zebra(+0xdc45)[0x4acc45] no thread information available
Zebra standard error output: