crash: invalid kernel virtual address: 0 type: "module symbols"

I've compiled a kernel on Gentoo with genkernel and saved a version of it with debugging symbols (and even its System.map but it doesn't seem to be needed for 'crash' because the output is identical with/without the map) then, I booted into this kernel with nokaslr kernel cmdline and triggered a manual crash (via sysrq+s,u,s then c), which triggered kexec to make a dump via makedumpfile and when done it rebooted back into the normal system. (Same kernel was used for normal and kexec: the stripped symbols version of it from /boot.)

click me to expand

``` -rw-r--r-- 1 root root 6181973 Mar 4 15:10 /boot/System.map-6.7.6-gentoo-x86_64 lrwxrwxrwx 1 root root 30 Mar 4 15:11 /boot/System.map -> System.map-6.7.6-gentoo-x86_64 ... -rw-r--r-- 1 root root 11858944 Mar 4 15:11 /boot/vmlinuz-6.7.6-gentoo-x86_64 lrwxrwxrwx 1 root root 27 Mar 4 15:11 /boot/kernel -> vmlinuz-6.7.6-gentoo-x86_64 ... -rw-r--r-- 1 root root 542623392 Mar 4 15:10 vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26 -rw------- 1 root root 62402000856 Mar 4 17:37 crashdump-2024-03-04-17_36_29 ```

Now I try to examine the crash dump like this:

click me to expand (to see output of this command) ``` # CRASHPAGER=/bin/cat crash -d1 -i autoexec --CRASHPAGER --mod /lib/modules/6.7.6-gentoo-x86_64/ vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26 crashdump-2024-03-04-17_36_29 ```

``` crash 8.0.4++ Copyright (C) 2002-2022 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2022 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. compressed kdump: header->utsname.machine: x86_64 diskdump_data: filename: crashdump-2024-03-04-17_36_29 flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED) dfd: 3 ofp: 0 machine_type: 62 (EM_X86_64) header: 5893c2d663e0 signature: "KDUMP " header_version: 6 utsname: sysname: Linux nodename: i87k release: 6.7.6-gentoo-x86_64 version: #1 SMP Mon Mar 4 15:10:10 CET 2024 machine: x86_64 domainname: (none) timestamp: tv_sec: 65e5f86b tv_usec: 0 status: 0 () block_size: 4096 sub_hdr_size: 2 bitmap_blocks: 1044 max_mapnr: 17100800 total_ram_blocks: 0 device_blocks: 0 written_blocks: 0 current_cpu: 0 nr_cpus: 12 tasks[nr_cpus]: 0 0 0 0 0 0 0 0 0 0 0 0 sub_header: 0 (n/a) sub_header_kdump: 5893c2d673f0 phys_base: 0 dump_level: 1 (0x1) (DUMP_EXCLUDE_ZERO) split: 0 start_pfn: (unused) end_pfn: (unused) offset_vmcoreinfo: 8496 (0x2130) size_vmcoreinfo: 3202 (0xc82) OSRELEASE=6.7.6-gentoo-x86_64 BUILD-ID=582684599a3cc125e8a53f43ce93a8a30d0f7ab2 PAGESIZE=4096 SYMBOL(init_uts_ns)=ffffffff837ebf60 OFFSET(uts_namespace.name)=0 SYMBOL(node_online_map)=ffffffff837f1268 SYMBOL(swapper_pg_dir)=ffffffff8302c000 SYMBOL(_stext)=ffffffff81000000 SYMBOL(vmap_area_list)=ffffffff8374e630 SYMBOL(mem_map)=ffffffff841fd030 SYMBOL(contig_page_data)=ffffffff837ec1c0 SYMBOL(mem_section)=ffff88904a7f6000 LENGTH(mem_section)=4096 SIZE(mem_section)=32 OFFSET(mem_section.section_mem_map)=0 NUMBER(SECTION_SIZE_BITS)=27 NUMBER(MAX_PHYSMEM_BITS)=46 SIZE(page)=64 SIZE(pglist_data)=7360 SIZE(zone)=1600 SIZE(free_area)=104 SIZE(list_head)=16 SIZE(nodemask_t)=8 OFFSET(page.flags)=0 OFFSET(page._refcount)=52 OFFSET(page.mapping)=24 OFFSET(page.lru)=8 OFFSET(page._mapcount)=48 OFFSET(page.private)=40 OFFSET(page.compound_head)=8 OFFSET(pglist_data.node_zones)=0 OFFSET(pglist_data.nr_zones)=6480 OFFSET(pglist_data.node_start_pfn)=6488 OFFSET(pglist_data.node_spanned_pages)=6504 OFFSET(pglist_data.node_id)=6512 OFFSET(zone.free_area)=192 OFFSET(zone.vm_stat)=1472 OFFSET(zone.spanned_pages)=136 OFFSET(free_area.free_list)=0 OFFSET(list_head.next)=0 OFFSET(list_head.prev)=8 OFFSET(vmap_area.va_start)=0 OFFSET(vmap_area.list)=40 LENGTH(zone.free_area)=11 SYMBOL(prb)=ffffffff83042270 SYMBOL(printk_rb_static)=ffffffff83042280 SYMBOL(clear_seq)=ffffffff841f48e0 SIZE(printk_ringbuffer)=88 OFFSET(printk_ringbuffer.desc_ring)=0 OFFSET(printk_ringbuffer.text_data_ring)=48 OFFSET(printk_ringbuffer.fail)=80 SIZE(prb_desc_ring)=48 OFFSET(prb_desc_ring.count_bits)=0 OFFSET(prb_desc_ring.descs)=8 OFFSET(prb_desc_ring.infos)=16 OFFSET(prb_desc_ring.head_id)=24 OFFSET(prb_desc_ring.tail_id)=32 SIZE(prb_desc)=24 OFFSET(prb_desc.state_var)=0 OFFSET(prb_desc.text_blk_lpos)=8 SIZE(prb_data_blk_lpos)=16 OFFSET(prb_data_blk_lpos.begin)=0 OFFSET(prb_data_blk_lpos.next)=8 SIZE(printk_info)=88 OFFSET(printk_info.seq)=0 OFFSET(printk_info.ts_nsec)=8 OFFSET(printk_info.text_len)=16 OFFSET(printk_info.caller_id)=20 OFFSET(printk_info.dev_info)=24 SIZE(dev_printk_info)=64 OFFSET(dev_printk_info.subsystem)=0 LENGTH(printk_info_subsystem)=16 OFFSET(dev_printk_info.device)=16 LENGTH(printk_info_device)=48 SIZE(prb_data_ring)=32 OFFSET(prb_data_ring.size_bits)=0 OFFSET(prb_data_ring.data)=8 OFFSET(prb_data_ring.head_lpos)=16 OFFSET(prb_data_ring.tail_lpos)=24 SIZE(atomic_long_t)=8 OFFSET(atomic_long_t.counter)=0 SIZE(latched_seq)=24 OFFSET(latched_seq.val)=8 LENGTH(free_area.free_list)=6 NUMBER(NR_FREE_PAGES)=0 NUMBER(PG_lru)=5 NUMBER(PG_private)=15 NUMBER(PG_swapcache)=12 NUMBER(PG_swapbacked)=19 NUMBER(PG_slab)=11 NUMBER(PG_head_mask)=64 NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129 NUMBER(PG_hugetlb)=8 NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257 SYMBOL(kallsyms_names)=ffffffff82a891c0 SYMBOL(kallsyms_num_syms)=ffffffff82a891b8 SYMBOL(kallsyms_token_table)=ffffffff82c222d0 SYMBOL(kallsyms_token_index)=ffffffff82c22680 SYMBOL(kallsyms_offsets)=ffffffff82c22880 SYMBOL(kallsyms_relative_base)=ffffffff82cc2e90 NUMBER(phys_base)=0 SYMBOL(init_top_pgt)=ffffffff8302c000 NUMBER(pgtable_l5_enabled)=0 KERNELOFFSET=0 NUMBER(KERNEL_IMAGE_SIZE)=1073741824 NUMBER(sme_mask)=0 CRASHTIME=1709570155 offset_note: 4200 (0x1068) size_note: 7500 (0x1d4c) notes_buf: 5893c2d5b440 num_vmcoredd_notes: 0 num_prstatus_notes: 12 notes[0]: 5893c2d5b440 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffa1f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffffff83003e38 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[1]: 5893c2d5b5a4 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffa5f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000a3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[2]: 5893c2d5b708 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffa9f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000abe90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[3]: 5893c2d5b86c (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffadf510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000b3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[4]: 5893c2d5b9d0 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffb1f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000bbe90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[5]: 5893c2d5bb34 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffb5f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000c3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[6]: 5893c2d5bc98 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffb9f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000cbe90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[7]: 5893c2d5bdfc (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffbdf510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000d3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[8]: 5893c2d5bf60 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffc1f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000dbe90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[9]: 5893c2d5c0c4 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffc5f510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000e3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[10]: 5893c2d5c228 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: 0 fpvalid: 0 R15: ffff888141cf9a38 R14: 0000000000000063 R13: 0000000000000001 R12: ffffffff82d78a21 RBP: ffffc90000304ce0 RBX: ffffc90000304bb0 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000000 RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffc90000304c58 RIP: ffffffff811b9f8c RFLAGS: 0000000000000046 RSP: ffffc90000304bb0 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 notes[11]: 5893c2d5c38c (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 ORIG_RAX: ffffffffffffffff fpvalid: 0 R15: 0000000000000000 R14: ffffffff83774660 R13: ffffffff83774678 R12: ffffffff83774320 RBP: 0000000000000008 RBX: ffffe8ffffcdf510 R11: 0000000000000000 R10: 0000000000000000 R9: 0000000000000000 R8: 0000000000000000 RAX: 0000000000000060 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000060 RDI: 0000000000000008 RIP: ffffffff822f6fa7 RFLAGS: 0000000000000046 RSP: ffffc900000f3e90 FS_BASE: 0000000000000000 GS_BASE: 0000000000000000 CS: 0010 SS: 0018 DS: 0000 ES: 0000 FS: 0000 GS: 0000 snapshot_task: 0 num_qemu_notes: 0 NOTE offsets: 1068 (NT_PRSTATUS) 11cc (NT_PRSTATUS) 1330 (NT_PRSTATUS) 1494 (NT_PRSTATUS) 15f8 (NT_PRSTATUS) 175c (NT_PRSTATUS) 18c0 (NT_PRSTATUS) 1a24 (NT_PRSTATUS) 1b88 (NT_PRSTATUS) 1cec (NT_PRSTATUS) 1e50 (NT_PRSTATUS) 1fb4 (NT_PRSTATUS) offset_eraseinfo: 0 (0x0) size_eraseinfo: 0 (0x0) start_pfn_64: (unused) end_pfn_64: (unused) max_mapnr_64: 17100800 (0x104f000) data_offset: 417000 block_size: 4096 block_shift: 12 bitmap: 75d08f803000 bitmap_len: 4276224 max_mapnr: 17100800 (0x104f000) dumpable_bitmap: 75d08fe00010 byte: 0 bit: 0 compressed_page: 5893c2da9450 curbufptr: 0 page_cache_hdr[0]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d99440 pg_hit_count: 0 page_cache_hdr[1]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9a440 pg_hit_count: 0 page_cache_hdr[2]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9b440 pg_hit_count: 0 page_cache_hdr[3]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9c440 pg_hit_count: 0 page_cache_hdr[4]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9d440 pg_hit_count: 0 page_cache_hdr[5]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9e440 pg_hit_count: 0 page_cache_hdr[6]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2d9f440 pg_hit_count: 0 page_cache_hdr[7]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da0440 pg_hit_count: 0 page_cache_hdr[8]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da1440 pg_hit_count: 0 page_cache_hdr[9]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da2440 pg_hit_count: 0 page_cache_hdr[10]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da3440 pg_hit_count: 0 page_cache_hdr[11]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da4440 pg_hit_count: 0 page_cache_hdr[12]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da5440 pg_hit_count: 0 page_cache_hdr[13]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da6440 pg_hit_count: 0 page_cache_hdr[14]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da7440 pg_hit_count: 0 page_cache_hdr[15]: pg_flags: 0 () pg_addr: 0 pg_bufptr: 5893c2da8440 pg_hit_count: 0 page_cache_buf: 5893c2d99440 evict_index: 0 evictions: 0 accesses: 0 cached_reads: 0 valid_pages: 5893c2d5d1a0 total_valid_pages: 16348585 readmem: read_diskdump() VMCOREINFO: NUMBER(phys_base): 0 -> 0 program_name: crash program_path: crash program_version: 8.0.4++ gdb_version: 10.2 program_pid: 88994 prompt: "crash> " flags: 1800000200c22 (TTY|HASH|SCROLL|CMDLINE_IFILE|READLINE|DISKDUMP|DATADEBUG) flags2: 100402 (ELF_NOTES|VMCOREINFO|REDZONE) namelist: vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26 dumpfile: crashdump-2024-03-04-17_36_29 live_memsrc: (null) system_map: (null) namelist_debug: (null) debuginfo_file: (null) namelist_orig: (null) namelist_dbg_orig: (null) kvmdump_mapfile: (null) memory_module: (null) memory_device: /dev/crash machine_type: X86_64 editing_mode: vi nfd: -1 mfd: -1 kfd: -1 dfd: -1 confd: -2 home: /root command_line: orig_line: eoc_index: 0 readline: 0 my_tty: pts/5 debug: 1 debug_save: 0 console: (null) redhat_debug_loc: /usr/lib/debug/lib/modules pipefd[2]: 0,0 nullfp: 5893c2d5b240 stdpipe: 0 pipe: 0 ifile: 0 ofile: 0 ifile_pipe: 0 ifile_ofile: 0 args_ifile: 0 input_file: autoexec ifile_in_progress: 0 () ifile_offset: 0 runtime_ifile_cmd: (unused) scroll_command: SCROLL_NONE redirect: 0 stdpipe_pid: 0 pipe_pid: 0 pipe_shell_pid: 0 pipe_command: symfile: 0 symfile2: 0 tmpfile: 0 saved_fp: 0 tmp_fp: 0 tmpfile2: 0 cmd_table: linux_command_table curcmd: crash lastcmd: (null) cur_gdb_cmd: 0 last_gdb_cmd: 0 cur_req: 0 cmdgencur: 0 curcmd_flags: 0 () curcmd_private: 0 cmd_cleanup: 0 cmd_cleanup_arg: 0 sigint_cnt: 0 sigaction: 5893c297bca8 gdb_sigaction: 5893c297bd40 main_loop_env: 5893c297bdd8 foreach_loop_env: 5893c297bea0 termios_orig: 5893c297bf68 termios_raw: 5893c297bfa4 ncmds: 0 cmdlist: 0 cmdlistsz: 0 output_radix: 0 (???) server: (null) server_pid: 0 port: 0 sockfd: 0 server_memsrc: (null) server_namelist: (null) rmfd: 0 rkfd: 0 rcvbufsize: 0 readmem: read_diskdump() writemem: write_diskdump() dumpfile memory: 0 curext: 0 sbrk: 5893c2dca000 cleanup: (null) scope: 0 (not set) nr_hash_queues: 32768 read_vmcoreinfo: 5893c2050bb0 error_fp: 75d0905f75c0 error_path: default gdb vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26 GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... State of pagination is off. crash: CONFIG_NR_CPUS: 64 crash: CONFIG_HZ: 300 crash: # CONFIG_DEBUG_INFO_REDUCED is not set cpu_possible_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 cpu_present_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 cpu_online_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 cpu_active_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 xtime timespec.tv_sec: 65e5f86b: Mon Mar 4 17:35:55 CET 2024 utsname: sysname: Linux nodename: i87k release: 6.7.6-gentoo-x86_64 version: #1 SMP Mon Mar 4 15:10:10 CET 2024 machine: x86_64 domainname: (none) base kernel version: 6.7.6 verify_namelist: dumpfile /proc/version: Linux version 6.7.6-gentoo-x86_64 (_kernelbuilder@i87k) (gcc (Gentoo 13.2.1_p20240210 p13) 13.2.1 20240210, GNU ld (Gentoo 2.42 p3) 2.42.0) #1 SMP Mon Mar 4 15:10:10 CET 2024 vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26: Linux version 6.7.6-gentoo-x86_64 (_kernelbuilder@i87k) (gcc (Gentoo 13.2.1_p20240210 p13) 13.2.1 20240210, GNU ld (Gentoo 2.42 p3) 2.42.0) #1 SMP Mon Mar 4 15:10:10 CET 2024 crash: get_cpus_present: present: 12 hypervisor: (undetermined) crash: per_cpu_symbol_search(hardirq_stack_ptr): NULL crash: per_cpu_symbol_search(per_cpu__irq_stack_union): NULL crash: per_cpu_symbol_search(per_cpu__cpu_number): NULL crash: per_cpu_symbol_search(per_cpu__current_task): NULL crash: get_cpus_present: present: 12 crash: per_cpu_symbol_search(per_cpu__init_tss): NULL crash: per_cpu_symbol_search(per_cpu__cpu_tss): NULL crash: per_cpu_symbol_search(per_cpu__orig_ist): NULL exception IST: 3: asm_exc_debug 2: asm_exc_nmi 1: asm_exc_double_fault 4: asm_exc_machine_check exception stacks: [0]: DOUBLEFAULT [1]: NMI [2]: DEBUG [3]: MCE [4]: (unknown) [5]: (unknown) [6]: (unknown) kernel NR_CPUS: 64 PAGESIZE=4096 mem_section_size = 32768 NR_SECTION_ROOTS = 4096 NR_MEM_SECTIONS = 524288 SECTIONS_PER_ROOT = 128 SECTION_ROOT_MASK = 0x7f PAGES_PER_SECTION = 32768 node_online_map: [1] -> nodes online: 1 node_table[0]: id: 0 pgdat: ffffffff837ec1c0 size: 17100799 present: 16741801 mem_map: ffffea0000000040 start_paddr: 1000 start_mapnr: 1 NOTE: page_hash_table does not exist in this kernel please wait... (gathering kmem slab cache data) kmem_cache_downsize: 208 to 208 pageflags from enum: 00000001 locked 00000002 writeback 00000004 referenced 00000008 uptodate 00000010 dirty 00000020 lru 00000040 head 00000080 waiters 00000100 active 00000200 workingset 00000400 error 00000800 slab 00001000 owner_priv_1 00002000 arch_1 00004000 reserved 00008000 private 00010000 private_2 00020000 mappedtodisk 00040000 reclaim 00080000 swapbacked 00100000 unevictable 00200000 mlocked 00400000 uncached 00800000 young 01000000 idle NOTE: unwind_table structure has changed, or does not exist in this kernel init_unwind_table: DWARF_UNWIND_EH_FRAME crash: struct module_memory detected. please wait... (gathering module symbol data) crash: invalid kernel virtual address: 0 type: "module symbols" ```

(if I don't specify nokaslr that error looks like: crash: invalid kernel virtual address: 19 type: "module gpl symbols", although, this was same version kernel it was a different compilation of it, maybe that's why 'gpl' is extra, unsure. I haven't tested without nokaslr in cmdline in this recompiled kernel that this whole issue is about)

that 'crash' I used was compiled on Gentoo from latest commit 5977936c0a91b89e48d026867e6a2f8261ba0c2d and had a little patch(why the extra output):

show 'crash' args and stuff at startup, helps knowing what it sees.

Index: /var/tmp/portage/dev-util/crash-9999/work/crash-9999/main.c
===================================================================
--- .orig/var/tmp/portage/dev-util/crash-9999/work/crash-9999/main.c
+++ crash-9999/main.c
@@ -713,6 +713,7 @@ main(int argc, char **argv)
    machdep_init(PRE_GDB);
         datatype_init();

+        dump_program_context();
    /*
     *  gdb_main_loop() modifies "command_loop_hook" to point to the 
          *  main_loop() function below, and then calls gdb's main() function.

I'm not sure if --mod dir is required, but the modules are here:

click to expand

``` /lib/modules/6.7.6-gentoo-x86_64/: total 560 drwxr-xr-x 1 root root 68 Mar 4 15:11 kernel drwxr-xr-x 1 root root 2786 Mar 4 15:14 .. drwxr-xr-x 1 root root 72 Mar 4 15:15 misc drwxr-xr-x 1 root root 460 Mar 4 15:15 . -rw-r--r-- 1 root root 4876 Mar 4 15:11 modules.order -rw-r--r-- 1 root root 236449 Mar 4 15:11 modules.builtin.modinfo -rw-r--r-- 1 root root 26143 Mar 4 15:11 modules.builtin lrwxrwxrwx 1 root root 21 Mar 4 15:11 build -> /tmp/genkernel.outdir -rw-r--r-- 1 root root 61961 Mar 4 15:15 modules.symbols -rw-r--r-- 1 root root 269 Mar 4 15:15 modules.softdep -rw-r--r-- 1 root root 15400 Mar 4 15:15 modules.dep.bin -rw-r--r-- 1 root root 9759 Mar 4 15:15 modules.dep -rw-r--r-- 1 root root 31304 Mar 4 15:15 modules.alias.bin -rw-r--r-- 1 root root 27328 Mar 4 15:15 modules.alias -rw-r--r-- 1 root root 69065 Mar 4 15:15 modules.symbols.bin -rw-r--r-- 1 root root 30997 Mar 4 15:15 modules.builtin.bin -rw-r--r-- 1 root root 25146 Mar 4 15:15 modules.builtin.alias.bin -rw-r--r-- 1 root root 126 Mar 4 15:15 modules.devname ```

autoexec file contents:

bt
quit

To get rid of the error, I've found that I can add --no_modules but this is just a mitigation, doesn't solve the problem, and it shows that the issue isn't just for the modules, as bt shows another similar error about something else:

# CRASHPAGER=/bin/cat crash -i autoexec --CRASHPAGER --no_modules --mod /lib/modules/6.7.6-gentoo-x86_64/ vmlinux-6.7.6-gentoo-x86_64-2024-03-04-15_14_26 crashdump-2024-03-04-17_36_29

click me to expand

crash-utility / crash

crash: invalid kernel virtual address: 0 type: "module symbols" #174