Not accessible address returned for a kmem_cache object when kernel built with KASAN or kfence

[thealberto]

Hi, I have manually fixed #1 for my environment but still I cannot use the sblist command.

I have printed kmem_cache_type and slab_caches from inside iter_slab_caches

Obviously the address associated with the cache isn't readable in my QEMU machine. Do you have an idea on how to solve it?

I'm using the kernel 6.2.0-rc3-next-20230112

/usr/local/bin/gdb -q vmlinux-kasan-bpf -ex "target remote :1234" -ex "source libslub/libslub.py" -ex "sblist"
GEF for linux ready, type `gef' to start, `gef config' to configure
90 commands loaded and 5 functions added for GDB 13.1 in 0.00ms using Python engine 3.10
Reading symbols from vmlinux-kasan-bpf...
Remote debugging using :1234
0xffffffff8a00464f in kfence_metadata ()

[ Legend: Modified register | Code | Heap | Stack | String ]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── registers ────
[!] Command 'registers' failed to execute properly, reason: [Errno 13] Permission denied: '/proc/1/maps'
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── stack ────
[!] Command 'dereference' failed to execute properly, reason: [Errno 13] Permission denied: '/proc/1/maps'
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── code:x86:64 ────
   0xffffffff8a004646 <kfence_metadata+207078> verw   WORD PTR [rip+0x463ef3]        # 0xffffffff8a468540
   0xffffffff8a00464d <kfence_metadata+207085> sti    
   0xffffffff8a00464e <kfence_metadata+207086> hlt    
 → 0xffffffff8a00464f <kfence_metadata+207087> ret    
   ↳  0xffffffff8a004b34 <kfence_metadata+208340> cli    
      0xffffffff8a004b35 <kfence_metadata+208341> mov    rdi, 0xffffffff8a004b03
      0xffffffff8a004b3c <kfence_metadata+208348> call   0xffffffff89fd1ec0 <kfence_metadata+352>
      0xffffffff8a004b41 <kfence_metadata+208353> call   0xffffffff89fd2d20 <kfence_metadata+4032>
      0xffffffff8a004b46 <kfence_metadata+208358> mov    rdi, 0xffffffff8a004b03
      0xffffffff8a004b4d <kfence_metadata+208365> call   0xffffffff89fd1dc0 <kfence_metadata+96>
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── threads ────
[#0] Id 1, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#1] Id 2, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#2] Id 5, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#3] Id 3, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#4] Id 4, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#5] Id 6, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
[#6] Id 7, stopped 0xffffffff8a00464f in kfence_metadata (), reason: SIGTRAP
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── trace ────
[#0] 0xffffffff8a00464f → kfence_metadata()
[#1] 0xffffffff8a004b34 → kfence_metadata()
[#2] 0xffffffff8c495700 → add BYTE PTR [rax+0x0], al
[#3] 0xffffffff815e8b2c → fast_find_migrateblock(cc=0xffffffff8c495700)
[#4] 0xffffffff815e8b2c → isolate_migratepages(cc=0xffffffff8c495700)
[#5] 0xffffffff815e8b2c → compact_zone(cc=0x41b58aa3, capc=0xe880d17900000000)
[#6] 0x2 → fixed_percpu_data()
[#7] 0xffffffff89fd3a5d → kfence_metadata()
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
name                    objs inuse slabs size obj_size objs_per_slab pages_per_slab
<class 'libslub.slub.sb.sb'>
Inside iter slab caches
struct kmem_cache
{
  next = 0x3c8003eac148dfff,
  prev = 0x1da850f0002
}

------------------------------- Exception raised -------------------------------
MemoryError: Cannot access memory at address 0x3c8003eac148dfff
----------------------------- Detailed stacktrace ------------------------------
......
gef

[thealberto]

Hi, I have tried with another kernel which does not use kasan and I was able to progress. Additionally my kernel isn't "stuck" in kfence_metadata`. Any suggestions?

Thanks

[thealberto]

I have recompiled the kernel without kfence and now the error is gone. Do you want to add it maybe into the docs or is there a better solution?

Thanks

[saidelike]

When compiling the kernel with KASAN or kfence (https://docs.kernel.org/dev-tools/kfence.html), the kernel structures will be different and so libslub might not work. Obviously you could fix libslub so it actually supports also KASAN or kfence specificities and use pattern matching to make it generic enough.

[saidelike]

Another thing you might find useful is using --loglevel in order to print more debugging output

[saidelike]

If you want, feel free to add the kernel version you tested it on that you confirmed work in the table from https://github.com/nccgroup/libslub/blob/main/docs/SupportedVersions.md#linuxslub-versions. Feel free to also document that kasan/kfence are currently not supported (in a new KASAN/kfence section at the end of the same file?) and then do a Pull Request so it can be merged in the repo.

[thealberto]

@saidelike thanks for the information. I'll submit a PR ASAP.