Reproducible panic in jails when using multiple FIBs in host.

[thetzim]

Hi. First, let me thank you for this project. BSDRP is running our main firewalls since 5 years.

I accidentally ran in this problem :

I have an sshd running in fib 2 (which as default route on some admin interface) on main system. Hence, when login with said daemon, the shell runs on fib 2.

If, from this shell, jexec into jail and start a process using network, it will panic kernel. Using setfib 0 jexec ... does not.

Jails only have default fib (0).

cpuid = 13; apic id = 0d
fault virtual address   = 0x0
fault code      = supervisor read instruction, page not present
instruction pointer = 0x20:0x0
stack pointer           = 0x28:0xfffffe013e03ca08
frame pointer           = 0x28:0xfffffe013e03ca30
code segment        = base 0x0, limit 0xfffff, type 0x1b
            = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags    = interrupt enabled, resume, IOPL = 0
current process     = 49685 (drill)
rdi:    6000600191236 rsi:            110ac rdx:                0
rcx:               20  r8:                0  r9: fffff8013308ea00
rax: fffff80004f20b98 rbx: fffffe013e03cac8 rbp: fffffe013e03ca30
r10:               11 r11: fffffe010b06e020 r12: fffffe010b06e020
r13: fffff8011b8701a8 r14:                0 r15:                0
trap number     = 12
panic: page fault
cpuid = 13
time = 1675949685
KDB: stack backtrace:
#0 0xffffffff80bf80bd at kdb_backtrace+0x5d
#1 0xffffffff80baacbc at vpanic+0x17c
#2 0xffffffff80baab33 at panic+0x43
#3 0xffffffff8101ac19 at trap_fatal+0x409
#4 0xffffffff8101ac6f at trap_pfault+0x4f
#5 0xffffffff8101a200 at trap+0x270
#6 0xffffffff80ff1f88 at calltrap+0x8
#7 0xffffffff80d154cc at in_pcbladdr+0x7c
#8 0xffffffff80d151da at in_pcbconnect_setup+0x1ba
#9 0xffffffff80d50343 at udp_send+0x5f3
#10 0xffffffff80c4c0ed at sosend_dgram+0x33d
#11 0xffffffff80c4d01b at sosend+0x3b
#12 0xffffffff80c53ab2 at kern_sendit+0x192
#13 0xffffffff80c53dda at sendit+0xba
#14 0xffffffff80c53d0d at sys_sendto+0x4d
#15 0xffffffff8101b51c at amd64_syscall+0x10c
#16 0xffffffff80ff289b at fast_syscall_common+0xf8
Uptime: 15m44s

[ocochard]

Thank, I was able to reproduce this bug on a generic FreeBSD, so I've created a Problem Report here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269441

[ocochard]

There is a patch in review, I will release a new "night" BSDRP release once this patch will land FreeBSD source. https://reviews.freebsd.org/D38505

[thetzim]

The error might be confusing for my fellow sysadmins, but that would avoid panics. Note that in 13.1-RELEASE, jexec would default to fib 0.