file(1) crash in jemalloc(3) processing an ELF target when compiled with -O2

[rwatson]

In trying to run a statically linked CheriABI file(1):

diff --git a/usr.bin/file/Makefile b/usr.bin/file/Makefile
index 70c211b..9f302d7 100644
--- a/usr.bin/file/Makefile
+++ b/usr.bin/file/Makefile
@@ -24,6 +24,10 @@ SRCDIR=      ${.CURDIR}/../../contrib/file
 .PATH: ${SRCDIR}/src
 .PATH: ${SRCDIR}/doc

+WANT_CHERI=pure
+WANT_DUMP=yes
+NO_SHARED=yes
+
 PROG=  file

 MAGICPATH?=    /usr/share/misc

I encounter the following crash in jemalloc:

root@:~ # file /bin/cat
CHERI cause: ExcCode: 0x01 RegNum: $c01 (length violation)
$c00: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c01: v:1 s:0 p:7fff807d b:0000000120164380 l:0000000000000160 o:0 t:0
$c02: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:411060 t:0
$c03: v:1 s:0 p:7fff807d b:0000000000a00000 l:0000000000a00000 o:0 t:0
$c04: v:1 s:0 p:7fff807d b:0000000000a00000 l:0000000000a00000 o:0 t:0
$c05: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c06: v:1 s:0 p:7fff807d b:0000007fffffca10 l:0000000000000008 o:0 t:0
$c07: v:1 s:0 p:7fff807d b:0000007fffff4760 l:0000000000008000 o:0 t:0
$c08: v:1 s:0 p:7fff807d b:0000007fffffd738 l:0000000000000004 o:0 t:0
$c09: v:1 s:0 p:7fff807d b:0000007fffffd5bc l:0000000000000004 o:0 t:0
$c10: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c11: v:1 s:0 p:7fff807d b:0000007fff7ff000 l:00000000007ff3a0 o:0 t:0
$c12: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:1200c4418 t:0
$c13: v:1 s:0 p:00008055 b:0000007fffffd060 l:0000000000000040 o:0 t:0
$c14: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c15: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c16: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c17: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:120058298 t:0
$c18: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:418040 t:0
$c19: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c20: v:1 s:0 p:7fff807d b:0000000000a00000 l:0000000000a00000 o:0 t:0
$c21: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c22: v:1 s:0 p:7fff807d b:0000007fffffed05 l:0000000000000009 o:0 t:0
$c23: v:1 s:0 p:7fff807d b:00000001200fb690 l:000000000000001e o:0 t:0
$c24: v:1 s:0 p:7fff807d b:000000000041e000 l:00000000000001c0 o:0 t:0
$c26: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c31: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:1200c4568 t:0
Jul 17 09:27:14  kernel: USER_CHERI_EXCEPTION: pid 550 tid 100037 (file), uid 0: CP2 fault (type 0x32)
Jul 17 09:27:14  kernel: Trapframe Register Dump:
Jul 17 09:27:14  kernel: zero: 0        at: 0x3ffffffffffffffc  v0: 0   v1: 0x3ffffffffffffffc
Jul 17 09:27:14  kernel: a0: 0x160      a1: 0x120164380 a2: 0x120118140 a3: 0x100000000
Jul 17 09:27:14  kernel: a4: 0xffffffff a5: 0x120118130 a6: 0x7fffffffffffffff  a6: 0x3f
Jul 17 09:27:14  kernel: t0: 0x40       t1: 0x3 t2: 0xffffffffffffffff  t3: 0x5555555555555555
Jul 17 09:27:14  kernel: t8: 0x3333333333333333 t9: 0x1200c4418 s0: 0xa00000    s1: 0x100000
Jul 17 09:27:14  kernel: s2: 0  s3: 0x1 s4: 0x3 s5: 0xc8
Jul 17 09:27:14  kernel: s6: 0  s7: 0x1200fa340 k0: 0   k1: 0
Jul 17 09:27:14  kernel: gp: 0x120154090        sp: 0x7fe620    s8: 0x7fe620    ra: 0x7ff0dc
Jul 17 09:27:14  kernel: sr: 0x408084b3 mullo: 0x4038302820181008       mulhi: 0x8101820283038  badvaddr: 0x1200c4568
Jul 17 09:27:14  kernel: cause: 0x48    pc: 0x1200c4568

Signal 34 (core dumped)

A load is attempted relative to a capability with a length of 0x160, but the requested register offset from that is 0x3ffffffffffffffc.

From Qemu-CHERI's tracing facility:

--- User mode
0x00000001200c4418:  daddiu     sp,sp,-160
    Write sp = 00000000007fe620
0x00000001200c441c:  csd        s8,sp,152(c11)
    Memory Write [0000007fffffd6b8] = 00000000007fe6c0
0x00000001200c4420:  csd        gp,sp,144(c11)
    Memory Write [0000007fffffd6b0] = 0000000120154090
0x00000001200c4424:  csd        s0,sp,136(c11)
    Memory Write [0000007fffffd6a8] = 0000000000a00000
0x00000001200c4428:  csc        c20,sp,96(c11)
    Cap Memory Write [0000007fffffd680] = v:1 tps:00000000ffff00fa
    c:0000000000a00000 b:0000000000a00000 l:0000000000a00000
0x00000001200c442c:  csc        c19,sp,64(c11)
    Cap Memory Write [0000007fffffd660] = v:0 tps:0000000000000000
    c:0000000000000000 b:0000000000000000 l:0000000000000000
0x00000001200c4430:  csc        c18,sp,32(c11)
    Cap Memory Write [0000007fffffd640] = v:1 tps:00000000ffff00fa
    c:0000000000418040 b:0000000000000000 l:0000010000000000
0x00000001200c4434:  csc        c17,sp,0(c11)
    Cap Memory Write [0000007fffffd620] = v:1 tps:00000000ffff002e
    c:0000000120058298 b:0000000000000000 l:0000010000000000
0x00000001200c4438:  move       s8,sp
    Write s8 = 00000000007fe620
0x00000001200c443c:  cgetoffset t9,c12
    Write t9 = 00000001200c4418
0x00000001200c4440:  lui        at,0x9
    Write at = 0000000000090000
0x00000001200c4444:  daddu      at,at,t9
    Write at = 0000000120154418
0x00000001200c4448:  daddiu     gp,at,-904
0x00000001200c444c:  cfromptr   c1,c0,zero
    Write C01|v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000
             |o:0000000000000000 t:0
0x00000001200c4450:  csetoffset c1,c1,zero
0x00000001200c4454:  cne        at,c1,c3
    Write at = 0000000000000001
0x00000001200c4458:  beqz       at,0x1200c46dc
0x00000001200c445c:  nop
0x00000001200c4460:  cgetoffset v0,c3
    Write v0 = 0000000000000000
0x00000001200c4464:  dsrl       at,v0,0x1
    Write at = 0000000000000000
0x00000001200c4468:  or at,v0,at
0x00000001200c446c:  dsrl       v1,at,0x2
0x00000001200c4470:  or at,at,v1
0x00000001200c4474:  dsrl       v1,at,0x4
0x00000001200c4478:  or at,at,v1
0x00000001200c447c:  dsrl       v1,at,0x8
0x00000001200c4480:  or at,at,v1
0x00000001200c4484:  dsrl       v1,at,0x10
0x00000001200c4488:  or at,at,v1
0x00000001200c448c:  dsrl32     v1,at,0x0
0x00000001200c4490:  nor        at,at,v1
    Write at = ffffffffffffffff
0x00000001200c4494:  dsrl       v1,at,0x1
    Write v1 = 7fffffffffffffff
0x00000001200c4498:  lui        a0,0x5555
    Write a0 = 0000000055550000
0x00000001200c449c:  daddiu     a0,a0,21845
    Write a0 = 0000000055555555
0x00000001200c44a0:  dsll       a0,a0,0x10
    Write a0 = 0000555555550000
0x00000001200c44a4:  daddiu     a0,a0,21845
    Write a0 = 0000555555555555
0x00000001200c44a8:  dsll       a0,a0,0x10
    Write a0 = 5555555555550000
0x00000001200c44ac:  daddiu     a0,a0,21845
    Write a0 = 5555555555555555
0x00000001200c44b0:  and        v1,v1,a0
    Write v1 = 5555555555555555
0x00000001200c44b4:  dsubu      at,at,v1
    Write at = aaaaaaaaaaaaaaaa
0x00000001200c44b8:  lui        v1,0x3333
    Write v1 = 0000000033330000
0x00000001200c44bc:  daddiu     v1,v1,13107
    Write v1 = 0000000033333333
0x00000001200c44c0:  dsll       v1,v1,0x10
    Write v1 = 0000333333330000
0x00000001200c44c4:  daddiu     v1,v1,13107
    Write v1 = 0000333333333333
0x00000001200c44c8:  dsll       v1,v1,0x10
    Write v1 = 3333333333330000
0x00000001200c44cc:  daddiu     v1,v1,13107
    Write v1 = 3333333333333333
0x00000001200c44d0:  and        a0,at,v1
    Write a0 = 2222222222222222
0x00000001200c44d4:  dsrl       at,at,0x2
    Write at = 2aaaaaaaaaaaaaaa
0x00000001200c44d8:  and        at,at,v1
    Write at = 2222222222222222
0x00000001200c44dc:  daddu      at,a0,at
    Write at = 4444444444444444
0x00000001200c44e0:  dsrl       v1,at,0x4
    Write v1 = 0444444444444444
0x00000001200c44e4:  daddu      at,at,v1
    Write at = 4888888888888888
0x00000001200c44e8:  lui        v1,0xf0f
    Write v1 = 000000000f0f0000
0x00000001200c44ec:  daddiu     v1,v1,3855
    Write v1 = 000000000f0f0f0f
0x00000001200c44f0:  dsll       v1,v1,0x10
    Write v1 = 00000f0f0f0f0000
0x00000001200c44f4:  daddiu     v1,v1,3855
    Write v1 = 00000f0f0f0f0f0f
0x00000001200c44f8:  dsll       v1,v1,0x10
    Write v1 = 0f0f0f0f0f0f0000
0x00000001200c44fc:  daddiu     v1,v1,3855
    Write v1 = 0f0f0f0f0f0f0f0f
0x00000001200c4500:  and        at,at,v1
    Write at = 0808080808080808
0x00000001200c4504:  lui        v1,0x101
    Write v1 = 0000000001010000
0x00000001200c4508:  daddiu     v1,v1,257
    Write v1 = 0000000001010101
0x00000001200c450c:  dsll       v1,v1,0x10
    Write v1 = 0000010101010000
0x00000001200c4510:  daddiu     v1,v1,257
    Write v1 = 0000010101010101
0x00000001200c4514:  dsll       v1,v1,0x10
    Write v1 = 0101010101010000
0x00000001200c4518:  daddiu     v1,v1,257
    Write v1 = 0101010101010101
0x00000001200c451c:  dmult      at,v1
0x00000001200c4520:  mflo       at
    Write at = 4038302820181008
0x00000001200c4524:  dsrl32     at,at,0x18
    Write at = 0000000000000040
0x00000001200c4528:  daddiu     v1,zero,63
    Write v1 = 000000000000003f
0x00000001200c452c:  dsubu      at,v1,at
    Write at = ffffffffffffffff
0x00000001200c4530:  dsrl       at,at,0x2
    Write at = 3fffffffffffffff
0x00000001200c4534:  daddiu     v1,zero,1
    Write v1 = 0000000000000001
0x00000001200c4538:  dsll32     v1,v1,0x1e
    Write v1 = 4000000000000000
0x00000001200c453c:  lui        a0,0x0
    Write a0 = 0000000000000000
0x00000001200c4540:  daddu      a0,a0,gp
    Write a0 = 0000000120154090
0x00000001200c4544:  ld a0,-15744(a0)
    Memory Read [0000000120150310] = 0000000120118800
    Write a0 = 0000000120118800
0x00000001200c4548:  lui        a1,0x0
    Write a1 = 0000000000000000
0x00000001200c454c:  daddu      a1,a1,gp
    Write a1 = 0000000120154090
0x00000001200c4550:  ld a1,-14528(a1)
    Memory Read [00000001201507d0] = 0000000120164380
    Write a1 = 0000000120164380
0x00000001200c4554:  ld a0,0(a0)
    Memory Read [0000000120118800] = 0000000000000160
    Write a0 = 0000000000000160
0x00000001200c4558:  daddiu     v1,v1,-4
    Write v1 = 3ffffffffffffffc
0x00000001200c455c:  and        at,at,v1
    Write at = 3ffffffffffffffc
0x00000001200c4560:  cfromptr   c1,c0,a1
    Write C01|v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000
             |o:0000000120164380 t:0
0x00000001200c4564:  csetbounds c1,c1,a0
    Write C01|v:1 s:0 p:7fff807d b:0000000120164380 l:0000000000000160
             |o:0000000000000000 t:0
0x00000001200c4568:  clw        a0,at,68(c1)
mips_cpu_do_interrupt enter: PC 00000001200c4568 EPC 00000001200c4418 precise co
processor 2 exception
    Write BadVAddr = 00000001200c4568
--- Exception #18: precise coprocessor 2, vector ffffffff80000180
    Write C31|v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000
             |o:00000001200c4568 t:0
    Write Status = 00000000408084b3
    Write Cause = 0000000000000048
    Write EPC = 00000001200c4568
--- Kernel mode (ERL=0, KX=1)

$a1, loaded via $gp, is 0x120164380, and points here (in BSS or similar):

0000000120164380 <__je_chunks_rtree>:
        ...

The derivation of $at is a bit painful.

Some disassembled code around this trace:

00000001200c4418 <__je_huge_salloc>:
}

size_t
huge_salloc(const void *ptr)
{
   1200c4418:   67bdff60        daddiu  sp,sp,-160
   1200c441c:   ebcbe89b        csd     s8,sp,152(c11)
   1200c4420:   eb8be893        csd     gp,sp,144(c11)
   1200c4424:   ea0be88b        csd     s0,sp,136(c11)
   1200c4428:   fa8be806        csc     c20,sp,96(c11)
   1200c442c:   fa6be804        csc     c19,sp,64(c11)
   1200c4430:   fa4be802        csc     c18,sp,32(c11)
   1200c4434:   fa2be800        csc     c17,sp,0(c11)
   1200c4438:   03a0f025        move    s8,sp
   1200c443c:   49b96002        cgetoffset      t9,c12
   1200c4440:   3c010009        lui     at,0x9
   1200c4444:   0039082d        daddu   at,at,t9
   1200c4448:   643cfc78        daddiu  gp,at,-904
   1200c444c:   48810007        cfromptr        c1,c0,zero
JEMALLOC_INLINE unsigned
rtree_start_level(rtree_t *rtree, uintptr_t key)
{
        unsigned start_level;

        if (unlikely(key == 0))
   1200c4450:   49a10801        csetoffset      c1,c1,zero
   1200c4454:   49c108c1        cne     at,c1,c3
   1200c4458:   102000a0        beqz    at,1200c46dc <__je_huge_salloc+0x2c4>
   1200c445c:   00000000        nop
                return (rtree->height - 1);

        start_level = rtree->start_level[lg_floor(key) >>
   1200c4460:   49a21802        cgetoffset      v0,c3
   1200c4464:   0002087a        dsrl    at,v0,0x1
   1200c4468:   00410825        or      at,v0,at
   1200c446c:   000118ba        dsrl    v1,at,0x2
   1200c4470:   00230825        or      at,at,v1
   1200c4474:   0001193a        dsrl    v1,at,0x4
   1200c4478:   00230825        or      at,at,v1
   1200c447c:   00011a3a        dsrl    v1,at,0x8
   1200c4480:   00230825        or      at,at,v1
   1200c4484:   00011c3a        dsrl    v1,at,0x10
   1200c4488:   00230825        or      at,at,v1
   1200c448c:   0001183e        dsrl32  v1,at,0x0
   1200c4490:   00230827        nor     at,at,v1
   1200c4494:   0001187a        dsrl    v1,at,0x1
   1200c4498:   3c045555        lui     a0,0x5555
   1200c449c:   64845555        daddiu  a0,a0,21845
   1200c44a0:   00042438        dsll    a0,a0,0x10
   1200c44a4:   64845555        daddiu  a0,a0,21845
   1200c44a8:   00042438        dsll    a0,a0,0x10
   1200c44ac:   64845555        daddiu  a0,a0,21845
   1200c44b0:   00641824        and     v1,v1,a0
   1200c44b4:   0023082f        dsubu   at,at,v1
   1200c44b8:   3c033333        lui     v1,0x3333
   1200c44bc:   64633333        daddiu  v1,v1,13107
   1200c44c0:   00031c38        dsll    v1,v1,0x10
   1200c44c4:   64633333        daddiu  v1,v1,13107
   1200c44c8:   00031c38        dsll    v1,v1,0x10
   1200c44cc:   64633333        daddiu  v1,v1,13107
   1200c44d0:   00232024        and     a0,at,v1
   1200c44d4:   000108ba        dsrl    at,at,0x2
   1200c44d8:   00230824        and     at,at,v1
   1200c44dc:   0081082d        daddu   at,a0,at
   1200c44e0:   0001193a        dsrl    v1,at,0x4
   1200c44e4:   0023082d        daddu   at,at,v1
   1200c44e8:   3c030f0f        lui     v1,0xf0f
   1200c44ec:   64630f0f        daddiu  v1,v1,3855
   1200c44f0:   00031c38        dsll    v1,v1,0x10
   1200c44f4:   64630f0f        daddiu  v1,v1,3855
   1200c44f8:   00031c38        dsll    v1,v1,0x10
   1200c44fc:   64630f0f        daddiu  v1,v1,3855
   1200c4500:   00230824        and     at,at,v1
   1200c4504:   3c030101        lui     v1,0x101
   1200c4508:   64630101        daddiu  v1,v1,257
   1200c450c:   00031c38        dsll    v1,v1,0x10
   1200c4510:   64630101        daddiu  v1,v1,257
   1200c4514:   00031c38        dsll    v1,v1,0x10
   1200c4518:   64630101        daddiu  v1,v1,257
   1200c451c:   0023001c        dmult   at,v1
   1200c4520:   00000812        mflo    at
   1200c4524:   00010e3e        dsrl32  at,at,0x18
   1200c4528:   6403003f        daddiu  v1,zero,63
   1200c452c:   0061082f        dsubu   at,v1,at
   1200c4530:   000108ba        dsrl    at,at,0x2
   1200c4534:   64030001        daddiu  v1,zero,1
   1200c4538:   00031fbc        dsll32  v1,v1,0x1e
   1200c453c:   3c040000        lui     a0,0x0
   1200c4540:   009c202d        daddu   a0,a0,gp
   1200c4544:   dc84c280        ld      a0,-15744(a0)
   1200c4548:   3c050000        lui     a1,0x0
   1200c454c:   00bc282d        daddu   a1,a1,gp
   1200c4550:   dca5c740        ld      a1,-14528(a1)
   1200c4554:   dc840000        ld      a0,0(a0)
   1200c4558:   6463fffc        daddiu  v1,v1,-4
   1200c455c:   00230824        and     at,at,v1
   1200c4560:   48810147        cfromptr        c1,c0,a1
   1200c4564:   48210900        csetbounds      c1,c1,a0

# Boom at instruction 1200c4568 due to a bounds check loading a word from (at+0x68)(c1):
# at: 0x3ffffffffffffffc
# v1: 0x3ffffffffffffffc
# a0: 0x160
# a1: 0x120164380
#
# $c00: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
# $c01: v:1 s:0 p:7fff807d b:0000000120164380 l:0000000000000160 o:0 t:0
   1200c4568:   c881088e        clw     a0,at,68(c1)
        for (i = start_level, node = rtree_subtree_tryread(rtree, start_level);
            /**/; i++, node = child) {
                if (!dependent && unlikely(!rtree_node_valid(node)))
                        return (NULL);
                subkey = rtree_subkey(rtree, key, i);
                if (i == rtree->height - 1) {
   1200c456c:   c8610086        clw     v1,zero,64(c1)

Unfortunately I am unable to compile libc at -O0 due to a linker failure, so cannot easily compare results there:

===> lib/libc (all)
make[3]: /home/rnw24/obj/mips.mips64/home/rnw24/git/cheribsd/worldcheri/home/rnw24/git/cheribsd/lib/libc/.depend, 1: ignoring stale .depend for /home/rnw24/obj/mips.mips64/home/rnw24/git/cheribsd/tmp/usr/libcheri/libcompiler_rt.a
make[3]: /home/rnw24/obj/mips.mips64/home/rnw24/git/cheribsd/worldcheri/home/rnw24/git/cheribsd/lib/libc/.depend, 1: ignoring stale .depend for /home/rnw24/obj/mips.mips64/home/rnw24/git/cheribsd/tmp/usr/libcheri/libssp_nonshared.a
building shared library libc.so.7
/home/rnw24/sdk256/sdk/bin/clang -g -integrated-as --target=cheri-unknown-freebsd -msoft-float --sysroot=/home/rnw24/obj/mips.mips64/home/rnw24/git/cheribsd/tmp -mabi=sandbox -mxgot -cheri-linker  -Wl,--no-warn-mismatch -nodefaultlibs -Wl,-init=crt_init_globals -Wl,--version-script=Version.map  -shared -Wl,-x  -o libc.so.7.full -Wl,-soname,libc.so.7  `NM='nm' NMFLAGS='' lorder _fcntl.So _ioctl.So _open.So bt_close.So bt_conv.So bt_debug.So bt_delete.So bt_get.So bt_open.So bt_overflow.So bt_page.So bt_put.So bt_search.So bt_seq.So bt_split.So bt_utils.So db.So hash.So hash_bigkey.So hash_buf.So hash_func.So hash_log2.So hash_page.So ndbm.So mpool.So mpool-compat.So rec_close.So rec_delete.So rec_get.So rec_open.So rec_put.So rec_search.So rec_seq.So rec_utils.So creat.So gethostid.So getwd.So killpg.So sethostid.So setpgrp.So setrgid.So setruid.So sigcompat.So __getosreldate.So __pthread_mutex_init_calloc_cb_stub.So __xuname.So _once_stub.So _pthread_stubs.So _rand48.So _spinlock_stub.So _thread_init.So alarm.So arc4random.So assert.So auxv.So basename.So cap_sandboxed.So check_utility_compat.So clock.So clock_getcpuclockid.So closedir.So confstr.So crypt.So ctermid.So daemon.So devname.So dirfd.So dirname.So disklabel.So dlfcn.So drand48.So dup3.So elf_utils.So erand48.So err.So errlst.So errno.So exec.So fdevname.So feature_present.So fmtcheck.So fmtmsg.So fnmatch.So fpclassify.So frexp.So fstab.So ftok.So fts.So fts-compat.So ftw.So getbootfile.So getbsize.So getcap.So getcwd.So getdomainname.So getgrent.So getgrouplist.So gethostname.So getloadavg.So getlogin.So getmntinfo.So getnetgrent.So getosreldate.So getpagesize.So getpagesizes.So getpeereid.So getprogname.So getpwent.So getttyent.So getusershell.So getutxent.So getvfsbyname.So glob.So initgroups.So isatty.So isinf.So isnan.So jrand48.So lcong48.So libc_dlopen.So lockf.So lrand48.So mrand48.So nftw.So nice.So nlist.So nrand48.So opendir.So pause.So pmadvise.So popen.So posix_spawn.So psignal.So pututxline.So pw_scan.So raise.So readdir.So readpassphrase.So recvmmsg.So rewinddir.So scandir.So seed48.So seekdir.So semctl.So sendmmsg.So setdomainname.So sethostname.So setjmperr.So setmode.So setproctitle.So setprogname.So siginterrupt.So siglist.So signal.So sigsetops.So sleep.So srand48.So statvfs.So stringlist.So strtofflags.So sysconf.So sysctl.So sysctlbyname.So sysctlnametomib.So syslog.So telldir.So termios.So time.So times.So timezone.So ttyname.So ttyslot.So ualarm.So ulimit.So uname.So unvis-compat.So usleep.So utime.So utxdb.So valloc.So wait.So wait3.So waitpid.So waitid.So wordexp.So tls.So pwcache.So unvis.So vis.So cancelpoints_sem.So cancelpoints_sem_new.So infinity.So fabs.So ldexp.So _ctx_start.So _set_tp.So makecontext.So signalcontext.So sigsetjmp.So trivial-getcontextx.So _setjmp_c.So setjmp_c.So gmon.So mcount.So citrus_bcs.So citrus_bcs_strtol.So citrus_bcs_strtoul.So citrus_csmapper.So citrus_db.So citrus_db_factory.So citrus_db_hash.So citrus_esdb.So citrus_hash.So citrus_iconv.So citrus_lookup.So citrus_lookup_factory.So citrus_mapper.So citrus_memstream.So citrus_mmap.So citrus_module.So citrus_none.So citrus_pivot_factory.So citrus_prop.So citrus_stdenc.So bsd_iconv.So iconv_compat.So inet_addr.So inet_cidr_ntop.So inet_cidr_pton.So inet_lnaof.So inet_makeaddr.So inet_net_ntop.So inet_net_pton.So inet_neta.So inet_netof.So inet_network.So inet_ntoa.So inet_ntop.So inet_pton.So nsap_addr.So ev_streams.So ev_timers.So ascii.So big5.So btowc.So collate.So collcmp.So euc.So fix_grouping.So gb18030.So gb2312.So gbk.So ctype.So isctype.So iswctype.So ldpart.So lmessages.So lmonetary.So lnumeric.So localeconv.So mblen.So mbrlen.So mbrtowc.So mbsinit.So mbsnrtowcs.So mbsrtowcs.So mbtowc.So mbstowcs.So mskanji.So nextwctype.So nl_langinfo.So nomacros.So none.So rpmatch.So rune.So runetype.So setlocale.So setrunelocale.So table.So tolower.So toupper.So utf8.So wcrtomb.So wcsnrtombs.So wcsrtombs.So wcsftime.So wcstof.So wcstod.So wcstoimax.So wcstol.So wcstold.So wcstoll.So wcstombs.So wcstoul.So wcstoull.So wcstoumax.So wctob.So wctomb.So wctrans.So wctype.So wcwidth.So xlocale.So c16rtomb_iconv.So c32rtomb_iconv.So mbrtoc16_iconv.So mbrtoc32_iconv.So md5c.So ns_name.So ns_netint.So ns_parse.So ns_print.So ns_samedomain.So ns_ttl.So base64.So ether_addr.So eui64.So gai_strerror.So getaddrinfo.So gethostbydns.So gethostbyht.So gethostbynis.So gethostnamadr.So getifaddrs.So getifmaddrs.So getnameinfo.So getnetbydns.So getnetbyht.So getnetbynis.So getnetnamadr.So getproto.So getprotoent.So getprotoname.So getservent.So if_indextoname.So if_nameindex.So if_nametoindex.So ip6opt.So linkaddr.So map_v4v6.So name6.So ntoh.So nsdispatch.So nslexer.So nsparser.So nss_compat.So rcmd.So rcmdsh.So recv.So rthdr.So sctp_sys_calls.So send.So sockatmark.So sourcefilter.So vars.So nscache.So nscachedcli.So msgcat.So acl_branding.So acl_calc_mask.So acl_copy.So acl_compat.So acl_delete.So acl_delete_entry.So acl_entry.So acl_flag.So acl_free.So acl_from_text.So acl_from_text_nfs4.So acl_get.So acl_id_to_name.So acl_init.So acl_perm.So acl_set.So acl_strip.So acl_support.So acl_support_nfs4.So acl_to_text.So acl_to_text_nfs4.So acl_valid.So extattr.So mac.So mac_exec.So mac_get.So mac_set.So subr_acl_nfs4.So regcomp.So regerror.So regexec.So regfree.So herror.So h_errno.So mtctxres.So res_comp.So res_data.So res_debug.So res_findzonecut.So res_init.So res_mkquery.So res_mkupdate.So res_query.So res_send.So res_state.So res_update.So _flock_stub.So asprintf.So clrerr.So dprintf.So fclose.So fcloseall.So fdopen.So feof.So ferror.So fflush.So fgetc.So fgetln.So fgetpos.So fgets.So fgetwc.So fgetwln.So fgetws.So fileno.So findfp.So flags.So fmemopen.So fopen.So fprintf.So fpurge.So fputc.So fputs.So fputwc.So fputws.So fread.So freopen.So fscanf.So fseek.So fsetpos.So ftell.So funopen.So fvwrite.So fwalk.So fwide.So fwprintf.So fwscanf.So fwrite.So getc.So getchar.So getdelim.So getline.So gets.So getw.So getwc.So getwchar.So makebuf.So mktemp.So open_memstream.So open_wmemstream.So perror.So printf.So printf-pos.So putc.So putchar.So puts.So putw.So putwc.So putwchar.So refill.So remove.So rewind.So rget.So scanf.So setbuf.So setbuffer.So setvbuf.So snprintf.So sprintf.So sscanf.So stdio.So swprintf.So swscanf.So tempnam.So tmpfile.So tmpnam.So ungetc.So ungetwc.So vasprintf.So vdprintf.So vfprintf.So vfscanf.So vfwprintf.So vfwscanf.So vprintf.So vscanf.So vsnprintf.So vsprintf.So vsscanf.So vswprintf.So vswscanf.So vwprintf.So vwscanf.So wbuf.So wprintf.So wscanf.So wsetup.So xprintf.So xprintf_float.So xprintf_int.So xprintf_str.So xprintf_errno.So xprintf_hexdump.So xprintf_quote.So xprintf_time.So xprintf_vis.So asctime.So difftime.So localtime.So strftime.So strptime.So timelocal.So time32.So ffs.So memchr.So memcmp.So memset.So strcat.So strcpy.So strcspn.So strlen.So strncat.So strncmp.So strncpy.So strpbrk.So strsep.So strspn.So strstr.So swab.So bcmp.So bzero.So strchr.So strcmp.So strrchr.So bcopy.So memchr_c.So memcmp_c.So memcpy.So memcpy_c.So memcpy_c_tofrom.So memmove.So memmove_c.So memset_c.So strchr_c.So strcmp_c.So strnlen_c.So strncmp_c.So strncpy_c.So strtol_c.So trivial-vdso_tc.So clock_gettime.So gettimeofday.So __vdso_gettimeofday.So __error.So interposing_table.So futimens.So utimensat.So fcntl.So ioctl.So open.So _fcntl.So _ioctl.So _open.So accept.So accept4.So aio_suspend.So close.So connect.So fsync.So fork.So kevent.So msync.So nanosleep.So openat.So poll.So ppoll.So pselect.So read.So readv.So recvfrom.So recvmsg.So select.So sendmsg.So sendto.So setcontext.So sigprocmask.So sigsuspend.So sigtimedwait.So sigwait.So sigwaitinfo.So swapcontext.So wait4.So wait6.So write.So writev.So sigaction.So Ovfork.So cerror.So exect.So pipe.So ptrace.So syscall.So open.So link.So unlink.So chdir.So fchdir.So mknod.So chmod.So chown.So getpid.So mount.So unmount.So setuid.So getuid.So geteuid.So getpeername.So getsockname.So access.So chflags.So fchflags.So sync.So kill.So getppid.So dup.So getegid.So profil.So ktrace.So getgid.So setlogin.So acct.So sigaltstack.So ioctl.So reboot.So revoke.So symlink.So readlink.So execve.So umask.So chroot.So vadvise.So munmap.So mprotect.So madvise.So mincore.So getgroups.So setgroups.So getpgrp.So setpgid.So setitimer.So swapon.So getitimer.So getdtablesize.So dup2.So fcntl.So setpriority.So socket.So getpriority.So bind.So setsockopt.So listen.So getrusage.So getsockopt.So settimeofday.So fchown.So fchmod.So setreuid.So setregid.So rename.So flock.So mkfifo.So shutdown.So socketpair.So mkdir.So rmdir.So utimes.So adjtime.So setsid.So quotactl.So nlm_syscall.So nfssvc.So lgetfh.So getfh.So sysarch.So rtprio.So setfib.So ntp_adjtime.So setgid.So setegid.So seteuid.So stat.So fstat.So lstat.So pathconf.So fpathconf.So getrlimit.So setrlimit.So getdirentries.So __sysctl.So mlock.So munlock.So undelete.So futimes.So getpgid.So semget.So semop.So msgget.So msgsnd.So msgrcv.So shmat.So shmdt.So shmget.So clock_settime.So clock_getres.So ktimer_create.So ktimer_delete.So ktimer_settime.So ktimer_gettime.So ktimer_getoverrun.So ffclock_getcounter.So ffclock_setestimate.So ffclock_getestimate.So clock_getcpuclockid2.So ntp_gettime.So minherit.So rfork.So issetugid.So lchown.So aio_read.So aio_write.So lio_listio.So getdents.So lchmod.So netbsd_lchown.So lutimes.So netbsd_msync.So nstat.So nfstat.So nlstat.So preadv.So pwritev.So fhopen.So fhstat.So modnext.So modstat.So modfnext.So modfind.So kldload.So kldunload.So kldfind.So kldnext.So kldstat.So kldfirstmod.So getsid.So setresuid.So setresgid.So aio_return.So aio_cancel.So aio_error.So mlockall.So munlockall.So __getcwd.So sched_setparam.So sched_getparam.So sched_setscheduler.So sched_getscheduler.So sched_yield.So sched_get_priority_max.So sched_get_priority_min.So sched_rr_get_interval.So utrace.So kldsym.So jail.So sigpending.So __acl_get_file.So __acl_set_file.So __acl_get_fd.So __acl_set_fd.So __acl_delete_file.So __acl_delete_fd.So __acl_aclcheck_file.So __acl_aclcheck_fd.So extattrctl.So extattr_set_file.So extattr_get_file.So extattr_delete_file.So aio_waitcomplete.So getresuid.So getresgid.So kqueue.So extattr_set_fd.So extattr_get_fd.So extattr_delete_fd.So __setugid.So eaccess.So nmount.So __mac_get_proc.So __mac_set_proc.So __mac_get_fd.So __mac_get_file.So __mac_set_fd.So __mac_set_file.So kenv.So lchflags.So uuidgen.So sendfile.So mac_syscall.So getfsstat.So statfs.So fstatfs.So fhstatfs.So __mac_get_pid.So __mac_get_link.So __mac_set_link.So extattr_set_link.So extattr_get_link.So extattr_delete_link.So __mac_execve.So sigreturn.So getcontext.So swapoff.So __acl_get_link.So __acl_set_link.So __acl_delete_link.So __acl_aclcheck_link.So thr_create.So thr_exit.So thr_self.So thr_kill.So jail_attach.So extattr_list_fd.So extattr_list_file.So extattr_list_link.So ksem_timedwait.So thr_suspend.So thr_wake.So kldunloadf.So audit.So auditon.So getauid.So setauid.So getaudit.So setaudit.So getaudit_addr.So setaudit_addr.So auditctl.So _umtx_op.So thr_new.So sigqueue.So kmq_open.So kmq_setattr.So kmq_timedreceive.So kmq_timedsend.So kmq_notify.So kmq_unlink.So abort2.So thr_set_name.So aio_fsync.So rtprio_thread.So sctp_peeloff.So sctp_generic_sendmsg.So sctp_generic_sendmsg_iov.So sctp_generic_recvmsg.So pread.So pwrite.So mmap.So lseek.So truncate.So ftruncate.So thr_kill2.So shm_open.So shm_unlink.So cpuset.So cpuset_setid.So cpuset_getid.So cpuset_getaffinity.So cpuset_setaffinity.So faccessat.So fchmodat.So fchownat.So fexecve.So fstatat.So futimesat.So linkat.So mkdirat.So mkfifoat.So mknodat.So readlinkat.So renameat.So symlinkat.So unlinkat.So posix_openpt.So gssd_syscall.So jail_get.So jail_set.So jail_remove.So closefrom.So __semctl.So msgctl.So shmctl.So lpathconf.So __cap_rights_get.So cap_enter.So cap_getmode.So pdfork.So pdkill.So pdgetpid.So getloginclass.So setloginclass.So rctl_get_racct.So rctl_get_rules.So rctl_get_limits.So rctl_add_rule.So rctl_remove_rule.So posix_fallocate.So posix_fadvise.So cap_rights_limit.So cap_ioctls_limit.So cap_ioctls_get.So cap_fcntls_limit.So cap_fcntls_get.So bindat.So connectat.So chflagsat.So pipe2.So aio_mlock.So procctl.So numa_getaffinity.So numa_setaffinity.So _clock_gettime.So _gettimeofday.So _futimens.So _utimensat.So _exit.So _getlogin.So _accept.So _accept4.So _aio_suspend.So _close.So _connect.So _fsync.So _fork.So _kevent.So _msync.So _nanosleep.So _openat.So _poll.So _ppoll.So _pselect.So _read.So _readv.So _recvfrom.So _recvmsg.So _select.So _sendmsg.So _sendto.So _setcontext.So _sigprocmask.So _sigsuspend.So _sigtimedwait.So _sigwait.So _sigwaitinfo.So _swapcontext.So _wait4.So _wait6.So _write.So _writev.So _sigaction.So stack_protector.So stack_protector_compat.So auth_none.So auth_unix.So authunix_prot.So bindresvport.So clnt_bcast.So clnt_dg.So clnt_generic.So clnt_perror.So clnt_raw.So clnt_simple.So clnt_vc.So rpc_dtablesize.So getnetconfig.So getnetpath.So getrpcent.So getrpcport.So mt_misc.So pmap_clnt.So pmap_getmaps.So pmap_getport.So pmap_prot.So pmap_prot2.So pmap_rmt.So rpc_prot.So rpc_commondata.So rpc_callmsg.So rpc_generic.So rpc_soc.So rpcb_clnt.So rpcb_prot.So rpcb_st_xdr.So rpcsec_gss_stub.So svc.So svc_auth.So svc_dg.So svc_auth_unix.So svc_generic.So svc_raw.So svc_run.So svc_simple.So svc_vc.So auth_time.So auth_des.So authdes_prot.So des_crypt.So des_soft.So crypt_client.So key_call.So key_prot_xdr.So getpublickey.So svc_auth_des.So netname.So netnamer.So rpcdname.So rtime.So crypt_clnt.So crypt_xdr.So uuid_compare.So uuid_create.So uuid_create_nil.So uuid_equal.So uuid_from_string.So uuid_hash.So uuid_is_nil.So uuid_stream.So uuid_to_string.So xdr.So xdr_array.So xdr_float.So xdr_mem.So xdr_rec.So xdr_reference.So xdr_sizeof.So xdr_stdio.So softfloat.So fpgetround.So fpsetround.So fpgetmask.So fpsetmask.So fpgetsticky.So eqsf2.So nesf2.So gtsf2.So gesf2.So ltsf2.So lesf2.So negsf2.So eqdf2.So nedf2.So gtdf2.So gedf2.So ltdf2.So ledf2.So negdf2.So unordsf2.So unorddf2.So xdryp.So yp_xdr.So yplib.So subr_capability.So machdep_ldisd.So _hdtoa.So _hldtoa.So _ldtoa.So glue.So gdtoa_dmisc.So gdtoa_dtoa.So gdtoa_gdtoa.So gdtoa_gethex.So gdtoa_gmisc.So gdtoa_hd_init.So gdtoa_hexnan.So gdtoa_misc.So gdtoa_smisc.So gdtoa_strtod.So gdtoa_strtodg.So gdtoa_strtof.So gdtoa_strtord.So gdtoa_sum.So gdtoa_ulp.So modf.So _Exit.So a64l.So abort.So abs.So atexit.So atof.So atoi.So atol.So atoll.So bsearch.So div.So exit.So getenv.So getopt.So getopt_long.So getsubopt.So hcreate.So hcreate_r.So hdestroy_r.So heapsort.So heapsort_b.So hsearch_r.So imaxabs.So imaxdiv.So insque.So l64a.So labs.So ldiv.So llabs.So lldiv.So lsearch.So merge.So mergesort_b.So ptsname.So qsort.So qsort_r.So quick_exit.So radixsort.So rand.So random.So reallocarray.So reallocf.So realpath.So remque.So strfmon.So strtoimax.So strtol.So strtoll.So strtoq.So strtoul.So strtonum.So strtoull.So strtoumax.So strtouq.So system.So tdelete.So tfind.So tsearch.So twalk.So jemalloc_jemalloc.So jemalloc_arena.So jemalloc_atomic.So jemalloc_base.So jemalloc_bitmap.So jemalloc_chunk.So jemalloc_chunk_dss.So jemalloc_chunk_mmap.So jemalloc_ckh.So jemalloc_ctl.So jemalloc_extent.So jemalloc_hash.So jemalloc_huge.So jemalloc_mb.So jemalloc_mutex.So jemalloc_pages.So jemalloc_prof.So jemalloc_quarantine.So jemalloc_rtree.So jemalloc_stats.So jemalloc_tcache.So jemalloc_tsd.So jemalloc_util.So bcmp.So bcopy.So bzero.So explicit_bzero.So ffs.So ffsl.So ffsll.So fls.So flsl.So flsll.So memccpy.So memchr.So memrchr.So memcmp.So memcpy.So memmem.So memmove.So memset.So stpcpy.So stpncpy.So strcasecmp.So strcat.So strcasestr.So strchr.So strchrnul.So strcmp.So strcoll.So strcpy.So strcspn.So strdup.So strerror.So strlcat.So strlcpy.So strlen.So strmode.So strncat.So strncmp.So strncpy.So strndup.So strnlen.So strnstr.So strpbrk.So strrchr.So strsep.So strsignal.So strspn.So strstr.So strtok.So strxfrm.So swab.So wcpcpy.So wcpncpy.So wcscasecmp.So wcscat.So wcschr.So wcscmp.So wcscoll.So wcscpy.So wcscspn.So wcsdup.So wcslcat.So wcslcpy.So wcslen.So wcsncasecmp.So wcsncat.So wcsncmp.So wcsncpy.So wcsnlen.So wcspbrk.So wcsrchr.So wcsspn.So wcsstr.So wcstok.So wcswidth.So wcsxfrm.So wmemchr.So wmemcmp.So wmemcpy.So wmemmove.So wmemset.So | tsort -q`  -lcompiler_rt  -lssp_nonshared
/home/rnw24/sdk256/sdk/bin/cheri-unknown-freebsd-ld: BFD 2.17.50 [FreeBSD] 2007-07-03 assertion fail /home/rnw24/sdk256/cheribsd/gnu/usr.bin/binutils/libbfd/../../../../contrib/binutils/bfd/elfxx-mips.c:7455
/home/rnw24/sdk256/sdk/bin/cheri-unknown-freebsd-ld: BFD 2.17.50 [FreeBSD] 2007-07-03 assertion fail /home/rnw24/sdk256/cheribsd/gnu/usr.bin/binutils/libbfd/../../../../contrib/binutils/bfd/elfxx-mips.c:2767
/home/rnw24/sdk256/sdk/bin/cheri-unknown-freebsd-ld: final link failed: Bad value
clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.
make[3]: stopped in /home/rnw24/git/cheribsd/lib/libc
*** Error code 1

(Also potentially of interest to @brooksdavis.)

[rwatson]

(Identical crashes occur on both Qemu-CHERI and a recent 256-bit bitfile, so this is not obviously an emulation/ISA-level problem.)

[rwatson]

Sadly, or perhaps happily, shifting to -O1:

diff --git a/share/mk/bsd.cheri.mk b/share/mk/bsd.cheri.mk
index 08589f47..ae2eef1 100644
--- a/share/mk/bsd.cheri.mk
+++ b/share/mk/bsd.cheri.mk
@@ -31,7 +31,7 @@ OBJCOPY:=     elfcopy
 _CHERI_CC+=    -mabi=sandbox -mxgot
 LIBDIR:=       /usr/libcheri
 ROOTOBJDIR=    ${.OBJDIR:S,${.CURDIR},,}${SRCTOP}/worldcheri${SRCTOP}
-CFLAGS+=       -O2 -ftls-model=local-exec
+CFLAGS+=       -O1 -ftls-model=local-exec
 .if ${MK_CHERI_LINKER} == "yes"
 _CHERI_CC+=    -cheri-linker
 CFLAGS+=       -Wno-error

keeps what appears to be the same crash:

root@:~ # file /bin/cat
CHERI cause: ExcCode: 0x01 RegNum: $c18 (length violation)
$c00: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c01: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c02: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:12015a580 t:0
$c03: v:1 s:0 p:7fff807d b:000000012015a580 l:0000000000000160 o:0 t:0
$c04: v:1 s:0 p:7fff807d b:0000000000c00000 l:0000000000a00000 o:0 t:0
$c05: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:41e000 t:0
$c06: v:1 s:0 p:7fff807d b:0000007fffffcb7c l:0000000000000001 o:0 t:0
$c07: v:1 s:0 p:7fff807d b:0000007fffffd91c l:0000000000000004 o:0 t:0
$c08: v:1 s:0 p:7fff807d b:0000007fffffd918 l:0000000000000004 o:0 t:0
$c09: v:1 s:0 p:7fff807d b:0000007fffffd79c l:0000000000000004 o:0 t:0
$c10: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c11: v:1 s:0 p:7fff807d b:0000007fff7ff000 l:00000000007ff3a0 o:0 t:0
$c12: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:1200b6bc8 t:0
$c13: v:1 s:0 p:00008055 b:0000007fffffd220 l:0000000000000040 o:0 t:0
$c14: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c15: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c16: v:0 s:0 p:00000000 b:0000000000000000 l:0000000000000000 o:0 t:0
$c17: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:1200b7240 t:0
$c18: v:1 s:0 p:7fff807d b:000000012015a580 l:0000000000000160 o:0 t:0
$c19: v:1 s:0 p:7fff807d b:0000000000c00000 l:0000000000a00000 o:0 t:0
$c20: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:41e000 t:0
$c21: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c22: v:1 s:0 p:7fff807d b:0000000000a186c0 l:0000000000140000 o:0 t:0
$c23: v:1 s:0 p:7fff807d b:0000007fffffed05 l:0000000000000009 o:0 t:0
$c24: v:1 s:0 p:7fff807d b:000000000043f000 l:00000000000001c0 o:0 t:0
$c26: v:1 s:0 p:7fff807d b:0000000000000000 l:0000010000000000 o:0 t:0
$c31: v:1 s:0 p:7fff8017 b:0000000000000000 l:0000010000000000 o:1200b7254 t:0
Jul 18 12:26:43  kernel: USER_CHERI_EXCEPTION: pid 546 tid 100046 (file), uid 0: CP2 fault (type 0x32)
Jul 18 12:26:43  kernel: Trapframe Register Dump:
Jul 18 12:26:43  kernel: zero: 0        at: 0x3ffffffffffffffc  v0: 0x3ffffffffffffffc  v1: 0x2222222222222222
Jul 18 12:26:43  kernel: a0: 0  a1: 0xa a2: 0x64        a3: 0x1
Jul 18 12:26:43  kernel: a4: 0x101010101010101  a5: 0x8000000000000000  a6: 0x3f38302820181008  a6: 0x23
Jul 18 12:26:43  kernel: t0: 0x1b       t1: 0   t2: 0   t3: 0
Jul 18 12:26:43  kernel: t8: 0xa        t9: 0x1200b71f0 s0: 0x1 s1: 0x120149050
Jul 18 12:26:43  kernel: s2: 0x120149050        s3: 0x3 s4: 0   s5: 0
Jul 18 12:26:43  kernel: s6: 0xc8       s7: 0x1200e9078 k0: 0   k1: 0
Jul 18 12:26:43  kernel: gp: 0x120149050        sp: 0x7fe660    s8: 0x7fe660    ra: 0x8
Jul 18 12:26:43  kernel: sr: 0x408084b3 mullo: 0x4038302820181008       mulhi: 0x8101820283038  badvaddr: 0x1200b7254
Jul 18 12:26:43  kernel: cause: 0x48    pc: 0x1200b7254
Signal 34 (core dumped)

Although the code is compiled pretty differently -- perhaps more readably:

00000001200b71f0 <__je_rtree_start_level>:
#endif

#if (defined(JEMALLOC_ENABLE_INLINE) || defined(JEMALLOC_RTREE_C_))
JEMALLOC_INLINE unsigned
rtree_start_level(rtree_t *rtree, uintptr_t key)
{
   1200b71f0:   67bdffa0        daddiu  sp,sp,-96
   1200b71f4:   ebcbe85b        csd     s8,sp,88(c11)
   1200b71f8:   fa4be802        csc     c18,sp,32(c11)
   1200b71fc:   fa2be800        csc     c17,sp,0(c11)
   1200b7200:   03a0f025        move    s8,sp
   1200b7204:   49b96002        cgetoffset      t9,c12
   1200b7208:   3c010009        lui     at,0x9
   1200b720c:   0039102d        daddu   v0,at,t9
   1200b7210:   48810007        cfromptr        c1,c0,zero
        unsigned start_level;

        if (unlikely(key == 0))
   1200b7214:   49a10801        csetoffset      c1,c1,zero
   1200b7218:   49c10901        cne     at,c1,c4
   1200b721c:   10200014        beqz    at,1200b7270 <__je_rtree_start_level+0x8
0>
   1200b7220:   49b21800        cmove   c18,c3
   1200b7224:   64411e60        daddiu  at,v0,7776
                return (rtree->height - 1);

        start_level = rtree->start_level[lg_floor(key) >>
   1200b7228:   3c020000        lui     v0,0x0
   1200b722c:   0041082d        daddu   at,v0,at
   1200b7230:   dc21b7e8        ld      at,-18456(at)
   1200b7234:   480c09ff        cgetpccsetoffset        c12,at
   1200b7238:   48f16000        cjalr   c12,c17
   1200b723c:   49a42002        cgetoffset      a0,c4  
   1200b7240:   000208ba        dsrl    at,v0,0x2
   1200b7244:   64020001        daddiu  v0,zero,1
   1200b7248:   000217bc        dsll32  v0,v0,0x1e
   1200b724c:   6442fffc        daddiu  v0,v0,-4
   1200b7250:   00220824        and     at,at,v0
   1200b7254:   c852088e        clw     v0,at,68(c18)  
            LG_RTREE_BITS_PER_LEVEL];
        assert(start_level < rtree->height);
        return (start_level);
}
   1200b7258:   03c0e825        move    sp,s8
   1200b725c:   da2be800        clc     c17,sp,0(c11)  
   1200b7260:   da4be802        clc     c18,sp,32(c11)
   1200b7264:   cbcbe85b        cld     s8,sp,88(c11)
   1200b7268:   49008800        cjr     c17
   1200b726c:   67bd0060        daddiu  sp,sp,96
rtree_start_level(rtree_t *rtree, uintptr_t key)
{
        unsigned start_level;

        if (unlikely(key == 0))
                return (rtree->height - 1);  
   1200b7270:   c8320086        clw     at,zero,64(c18)
   1200b7274:   0802dc96        j       1200b7258 <__je_rtree_start_level+0x68>
   1200b7278:   2422ffff        addiu   v0,at,-1
   1200b727c:   00000000        nop

[davidchisnall]

That's good news. The optimisations that run at -O1 are not as complex (and, most happily, don't include GVN).

[davidchisnall]

Do you have a trace of this that shows what lg_floor returns?

[davidchisnall]

I believe that this might not actually be a compiler bug. It calls lg_floor(key), where key is a uintptr_t. This function attempts to count the leading zeros, but doesn't appear to have a case to deal with capabilities. If we ran the upstream autoconf stuff then I think that we'd see a compile failure because the macros for defining the size of a pointer would be set to something different from the ones that define the size of an integer or a long.

[brooksdavis]

It looks like lg_floor() is being called with an implicit cast of key to size_t which (assuming c4 is the pre-lg_floor() value should trigger an assert() so I'll need to look at a trace. I'll do that after I investigate a newly arisen crash in kyua.

LG_SIZEOF_PTR is a bit of an oddity in the jemalloc code. Except for the place where it is (rather pointlessly) used to define SIZEOF_PTR it's used as something that would be better spelt LG_RANGEOF_PTR.

[brooksdavis]

root@beri1:~ # ~ctsrd/file ~ctsrd/file
/home/ctsrd/file: ELF 64-bit MSB executable, MIPS with CHERI (unofficial), version 1 (FreeBSD), statically linked, for FreeBSD 11.0 (1100097), not stripped