search

crash-utility / crash

Linux kernel crash utility
https://crash-utility.github.io
819 stars 271 forks source link

WARNING: invalid linux_banner pointer: 65762078756e694c + vmlinux and /proc/kcore do not match! #160

Open vt-alt opened 10 months ago

vt-alt commented 10 months ago

crash 8.0.3 does not work on Linux 6.5.11:

# crash
crash 8.0.3-alt1
...
GNU gdb (GDB) 10.2
...
warning: Loadable section ".notes" outside of ELF segments

WARNING: invalid linux_banner pointer: 65762078756e694c
crash: /usr/lib/debug/lib/modules/6.5.11-un-def-alt1/vmlinux and /proc/kcore do not match!

Tail of the run with debug:

# crash -d9
...
GETBUF(1024 -> 0)
<readmem: ffffffff82e4f960, KVADDR, "possible", 1024, (ROE), 5578ac85e800>
<read_proc_kcore: addr: ffffffff82e4f960 paddr: 2e4f960 cnt: 1024>
cpu_possible_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
<readmem: ffffffff82e4f160, KVADDR, "present", 1024, (ROE), 5578ac85e800>
<read_proc_kcore: addr: ffffffff82e4f160 paddr: 2e4f160 cnt: 1024>
 cpu_present_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
<readmem: ffffffff82e4f560, KVADDR, "online", 1024, (ROE), 5578ac85e800>
<read_proc_kcore: addr: ffffffff82e4f560 paddr: 2e4f560 cnt: 1024>
  cpu_online_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
<readmem: ffffffff82e4ed60, KVADDR, "active", 1024, (ROE), 5578ac85e800>
<read_proc_kcore: addr: ffffffff82e4ed60 paddr: 2e4ed60 cnt: 672>
<read_proc_kcore: addr: ffffffff82e4f000 paddr: 2e4f000 cnt: 352>
  cpu_active_mask: cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
FREEBUF(0)
<readmem: ffffffff836ce760, KVADDR, "xen_start_info", 8, (ROE), 7fffd781bf68>
<read_proc_kcore: addr: ffffffff836ce760 paddr: 36ce760 cnt: 8>
<readmem: ffffffff8435fc90, KVADDR, "shadow_timekeeper xtime_sec", 8, (ROE), 7fffd781bf60>
<read_proc_kcore: addr: ffffffff8435fc90 paddr: 435fc90 cnt: 8>
xtime timespec.tv_sec: 655704cd: Fri Nov 17 09:14:37 MSK 2023
<readmem: ffffffff82e20ec0, KVADDR, "init_uts_ns", 390, (ROE), 5578ac84899c>
<read_proc_kcore: addr: ffffffff82e20ec0 paddr: 2e20ec0 cnt: 320>
<read_proc_kcore: addr: ffffffff82e21000 paddr: 2e21000 cnt: 70>
utsname:
     sysname: Linux
    nodename: REX
     release: 6.5.11-un-def-alt1
     version: #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023
     machine: x86_64
  domainname: (none)
base kernel version: 6.5.11
<readmem: ffffffff821368a0, KVADDR, "linux_banner", 8, (FOE), 7fffd7819c10>
<read_proc_kcore: addr: ffffffff821368a0 paddr: 21368a0 cnt: 8>
WARNING: invalid linux_banner pointer: 65762078756e694c
<readmem: 65762078756e694c, KVADDR, "accessible check", 8, (ROE|Q), 7fffd7819c18>
crash: invalid kernel virtual address: 65762078756e694c  type: "accessible check"
crash: /usr/lib/debug/lib/modules/6.5.11-un-def-alt1/vmlinux and /proc/kcore do not match!

Usage:

  crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS]     (dumpfile form)
  crash [OPTION]... [NAMELIST]                          (live system form)

Enter "crash -h" for details.
#

Proof that linux_banner is there and well:

# grep linux_banner /proc/kallsyms
ffffffff821368a0 D linux_banner
# gdb -q /usr/lib/debug/lib/modules/6.5.11-un-def-alt1/vmlinux /proc/kcore
Reading symbols from /usr/lib/debug/lib/modules/6.5.11-un-def-alt1/vmlinux...
Reading symbols from /usr/lib/debug/lib/modules/6.5.11-un-def-alt1/vmlinux.debug...
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-un-def root=UUID=56b25751-2d48-4138-99ac-7e48a8ecbf63'.
#0  0x0000000000000000 in fixed_percpu_data ()
(gdb) x/s linux_banner
0xffffffff821368a0 <linux_banner>:      "Linux version 6.5.11-un-def-alt1 (builder@localhost.localdomain) (gcc-13 (GCC) 13.2.1 20230817 (ALT Sisyphus 13.2.1-alt2), GNU ld (GNU Binutils) 2.41.0.20230826) #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:1"...

Sane version of crash works on Linux 6.1.62 without problem.

vt-alt commented 10 months ago

Forgot to show run for 8.0.4. It passes further but still fails:

# crash
crash 8.0.4-alt1
...
warning: Loadable section ".notes" outside of ELF segments

crash: cannot determine idle task addresses from init_tasks[] or runqueues[]

crash: cannot resolve "init_task_union"
#

Tail of run with debug:

# crash -d9
...
utsname:
     sysname: Linux
    nodename: REX
     release: 6.5.11-un-def-alt1
     version: #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023
     machine: x86_64
  domainname: (none)
base kernel version: 6.5.11
GETBUF(344 -> 0)
FREEBUF(0)
<readmem: ffffffff821368a0, KVADDR, "accessible check", 8, (ROE|Q), 7ffc4de2c138>
<read_proc_kcore: addr: ffffffff821368a0 paddr: 21368a0 cnt: 8>
<readmem: ffffffff821368a0, KVADDR, "read_string characters", 1499, (ROE|Q), 7ffc4de2cd00>
<read_proc_kcore: addr: ffffffff821368a0 paddr: 21368a0 cnt: 1499>
linux_banner:
Linux version 6.5.11-un-def-alt1 (builder@localhost.localdomain) (gcc-13 (GCC) 13.2.1 20230817 (ALT Sisyphus 13.2.1-alt2), GNU ld (GNU Binutils) 2.41.0.2023082
6) #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023

verify_namelist:
live memory /proc/version:
Linux version 6.5.11-un-def-alt1 (builder@localhost.localdomain) (gcc-13 (GCC) 13.2.1 20230817 (ALT Sisyphus 13.2.1-alt2), GNU ld (GNU Binutils) 2.41.0.20230826) #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023
...
ffffffffc21ed080 __func__.56
ffffffffc21ed1fc _note_15
ffffffffc21ed214 _note_14
ffffffffc21ed22c orc_header
ffffffffc21ee000 _MODULE_RODATA_END_xt_conntrack
<readmem: ffffffff84337e40, KVADDR, "nr_threads", 4, (FOE), 55ce4babda1c>
<read_proc_kcore: addr: ffffffff84337e40 paddr: 4337e40 cnt: 4>
crash: per_cpu_symbol_search(per_cpu__runqueues): NULL
crash: cannot determine idle task addresses from init_tasks[] or runqueues[]

crash: cannot resolve "init_task_union"
#
k-hagio commented 10 months ago

hmm, no runqueues found? I've not seen it on 6.5.

What does gdb print?

$ gdb vmlinux
(gdb) whatis runqueues
type = struct rq
(gdb) info address runqueues
Symbol "runqueues" is static storage at address 0x31d00.
vt-alt commented 10 months ago

Now on 6.5.12 (errors messages are the same):

# gdb -q /usr/lib/debug/lib/modules/$(uname -r)/vmlinux
Reading symbols from /usr/lib/debug/lib/modules/6.5.12-un-def-alt1/vmlinux...
Reading symbols from /usr/lib/debug/usr/lib/debug/lib/modules/6.5.12-un-def-alt1/vmlinux.debug...
(gdb) whatis runqueues
type = struct rq
(gdb) info address runqueues
Symbol "runqueues" is static storage at address 0x31e80.
(gdb) quit
# gdb --version
GNU gdb (GDB) 13.2.0.43.854f46b6377-alt3 (ALT Sisyphus)
k-hagio commented 10 months ago

Thanks, that looks sane... There may be something strange in per_cpu_symbol_search(), is it possible to check why it's not found?

I think these should be like this normally...

# nm vmlinux | grep -e ' __per_cpu_[es]' -e ' runqueues'
0000000000035000 D __per_cpu_end
0000000000000000 D __per_cpu_start
0000000000032140 D runqueues
vt-alt commented 10 months ago 
6.5.12-un-def-alt1:~# nm /usr/lib/debug/lib/modules/$(uname -r)/vmlinux | grep -e ' __per_cpu_[es]' -e ' runqueues'
nm: /usr/lib/debug/lib/modules/6.5.12-un-def-alt1/vmlinux: no symbols
6.5.12-un-def-alt1:~# nm /usr/lib/debug/lib/modules/$(uname -r)/vmlinux.debug | grep -e ' __per_cpu_[es]' -e ' runqueues'
0000000000035000 B __per_cpu_end
0000000000000000 B __per_cpu_start
0000000000031e80 B runqueues

Yes we don't have .symtab in vmlinux since Linux 6.5 (but it is present in vmlinux.debug). This is not intentional, though, it happened on its own that eu-strip started to strip it since 6.5 (we have elfutils-0.189-46-g27a84961). Same version of eu-strip leaving .symtab in vmlinux for 6.1.

How sections are look:

6.5.12-un-def-alt1:/usr/lib/debug/lib/modules/6.5.12-un-def-alt1# eu-readelf -S vmlinux
There are 43 section headers, starting at offset 0x42001f8:

Section Headers:
[Nr] Name                 Type         Addr             Off      Size     ES Flags Lk Inf Al
[ 0]                      NULL         0000000000000000 00000000 00000000  0        0   0  0
[ 1] .text                PROGBITS     ffffffff81000000 00200000 01000000  0 AX     0   0 4096
[ 2] .rodata              PROGBITS     ffffffff82000000 01200000 0054c076  0 WA     0   0 4096
[ 3] .pci_fixup           PROGBITS     ffffffff8254c080 0174c080 000037d0  0 A      0   0 16
[ 4] __ksymtab            PROGBITS     ffffffff8254f850 0174f850 00011d54  0 A      0   0  4
[ 5] __ksymtab_gpl        PROGBITS     ffffffff825615a4 017615a4 00011c88  0 A      0   0  4
[ 6] __kcrctab            PROGBITS     ffffffff8257322c 0177322c 00005f1c  0 A      0   0  1
[ 7] __kcrctab_gpl        PROGBITS     ffffffff82579148 01779148 00005ed8  0 A      0   0  1
[ 8] __ksymtab_strings    PROGBITS     ffffffff8257f020 0177f020 0003b2bc  1 AMS    0   0  1
[ 9] __init_rodata        PROGBITS     ffffffff825ba2e0 017ba2e0 000000b0  0 A      0   0 32
[10] __param              PROGBITS     ffffffff825ba390 017ba390 00003188  0 A      0   0  8
[11] __modver             PROGBITS     ffffffff825bd518 017bd518 00000288  0 WA     0   0  8
[12] __ex_table           PROGBITS     ffffffff825bd7a0 017bd7a0 00001b90  0 A      0   0  4
[13] .notes               NOTE         ffffffff825bf330 017bf330 000001d8  0 A      0   0  4
[14] .BTF                 PROGBITS     ffffffff825bf508 017bf508 004c872c  0 A      0   0  1
[15] .BTF_ids             PROGBITS     ffffffff82a87c34 01c87c34 00000a64  0 A      0   0  1
[16] .data                PROGBITS     ffffffff82c00000 01e00000 0025bf40  0 WA     0   0 8192
[17] __bug_table          PROGBITS     ffffffff82e5bf40 0205bf40 0002673c  0 WA     0   0  1
[18] .orc_header          PROGBITS     ffffffff82e8267c 0208267c 00000014  0 A      0   0  4
[19] .orc_unwind_ip       PROGBITS     ffffffff82e82690 02082690 001a6a38  0 A      0   0  1
[20] .orc_unwind          PROGBITS     ffffffff830290c8 022290c8 00279f54  0 A      0   0  1
[21] .orc_lookup          NOBITS       ffffffff832a301c 024a301c 00040004  0 WA     0   0  1
[22] .vvar                PROGBITS     ffffffff832e4000 024e4000 00001000  0 WA     0   0 16
[23] .data..percpu        PROGBITS     0000000000000000 02600000 00035000  0 WA     0   0 4096
[24] .init.text           PROGBITS     ffffffff8331a000 0271a000 0009b360  0 AX     0   0 16
[25] .altinstr_aux        PROGBITS     ffffffff833b5360 027b5360 0000154e  0 AX     0   0  1
[26] .init.data           PROGBITS     ffffffff833b8000 027b8000 001c0930  0 WA     0   0 8192
[27] .x86_cpu_dev.init    PROGBITS     ffffffff83578930 02978930 00000028  0 A      0   0  8
[28] .parainstructions    PROGBITS     ffffffff83578958 02978958 0000f280  0 A      0   0  8
[29] .retpoline_sites     PROGBITS     ffffffff83587bd8 02987bd8 00009708  0 A      0   0  1
[30] .return_sites        PROGBITS     ffffffff835912e0 029912e0 000392a8  0 A      0   0  1
[31] .call_sites          PROGBITS     ffffffff835ca588 029ca588 000e4c58  0 A      0   0  1
[32] .altinstructions     PROGBITS     ffffffff836af1e0 02aaf1e0 0000cc6c  0 A      0   0  1
[33] .altinstr_replacement PROGBITS     ffffffff836bbe4c 02abbe4c 00002517  0 AX     0   0  1
[34] .apicdrivers         PROGBITS     ffffffff836be368 02abe368 00000028  0 WA     0   0  8
[35] .exit.text           PROGBITS     ffffffff836be390 02abe390 00002337  0 AX     0   0 16
[36] .smp_locks           PROGBITS     ffffffff836c1000 02ac1000 0000a000  0 A      0   0  4
[37] .data_nosave         PROGBITS     ffffffff836cb000 02acb000 00001000  0 WA     0   0  4
[38] .bss                 NOBITS       ffffffff836cc000 02acc000 01134000  0 WA     0   0 4096
[39] .brk                 NOBITS       ffffffff84800000 02acc000 0002c000  0 WA     0   0  1
[40] .init.scratch        PROGBITS     ffffffff84a00000 03e00000 00400000  0 WA     0   0 32
[41] .gnu_debuglink       PROGBITS     0000000000000000 04200000 00000014  0        0   0  4
[42] .shstrtab            STRTAB       0000000000000000 04200014 000001e3  0        0   0  1

6.5.12-un-def-alt1:/usr/lib/debug/lib/modules/6.5.12-un-def-alt1# eu-readelf -S vmlinux.debug
There are 54 section headers, starting at offset 0x100c2da0:

Section Headers:
[Nr] Name                 Type         Addr             Off      Size     ES Flags Lk Inf Al
[ 0]                      NULL         0000000000000000 00000000 00000000  0        0   0  0
[ 1] .text                NOBITS       ffffffff81000000 00001000 01000000  0 AX     0   0 4096
[ 2] .rodata              NOBITS       ffffffff82000000 00001000 0054c076  0 WA     0   0 4096
[ 3] .pci_fixup           NOBITS       ffffffff8254c080 00001000 000037d0  0 A      0   0 16
[ 4] __ksymtab            NOBITS       ffffffff8254f850 00001000 00011d54  0 A      0   0  4
[ 5] __ksymtab_gpl        NOBITS       ffffffff825615a4 00001000 00011c88  0 A      0   0  4
[ 6] __kcrctab            NOBITS       ffffffff8257322c 00001000 00005f1c  0 A      0   0  1
[ 7] __kcrctab_gpl        NOBITS       ffffffff82579148 00001000 00005ed8  0 A      0   0  1
[ 8] __ksymtab_strings    NOBITS       ffffffff8257f020 00001000 0003b2bc  1 AMS    0   0  1
[ 9] __init_rodata        NOBITS       ffffffff825ba2e0 00001000 000000b0  0 A      0   0 32
[10] __param              NOBITS       ffffffff825ba390 00001000 00003188  0 A      0   0  8
[11] __modver             NOBITS       ffffffff825bd518 00001000 00000288  0 WA     0   0  8
[12] __ex_table           NOBITS       ffffffff825bd7a0 00001000 00001b90  0 A      0   0  4
[13] .notes               NOTE         ffffffff825bf330 00001000 000001d8  0 A      0   0  4
[14] .BTF                 NOBITS       ffffffff825bf508 000011d8 004c872c  0 A      0   0  1
[15] .BTF_ids             NOBITS       ffffffff82a87c34 000011d8 00000a64  0 A      0   0  1
[16] .data                NOBITS       ffffffff82c00000 00002000 0025bf40  0 WA     0   0 8192
[17] __bug_table          NOBITS       ffffffff82e5bf40 00002000 0002673c  0 WA     0   0  1
[18] .orc_header          NOBITS       ffffffff82e8267c 00002000 00000014  0 A      0   0  4
[19] .orc_unwind_ip       NOBITS       ffffffff82e82690 00002000 001a6a38  0 A      0   0  1
[20] .orc_unwind          NOBITS       ffffffff830290c8 00002000 00279f54  0 A      0   0  1
[21] .orc_lookup          NOBITS       ffffffff832a301c 00002000 00040004  0 WA     0   0  1
[22] .vvar                NOBITS       ffffffff832e4000 00002000 00001000  0 WA     0   0 16
[23] .data..percpu        NOBITS       0000000000000000 00002000 00035000  0 WA     0   0 4096
[24] .init.text           NOBITS       ffffffff8331a000 00002000 0009b360  0 AX     0   0 16
[25] .altinstr_aux        NOBITS       ffffffff833b5360 00002000 0000154e  0 AX     0   0  1
[26] .init.data           NOBITS       ffffffff833b8000 00002000 001c0930  0 WA     0   0 8192
[27] .x86_cpu_dev.init    NOBITS       ffffffff83578930 00002000 00000028  0 A      0   0  8
[28] .parainstructions    NOBITS       ffffffff83578958 00002000 0000f280  0 A      0   0  8
[29] .retpoline_sites     NOBITS       ffffffff83587bd8 00002000 00009708  0 A      0   0  1
[30] .return_sites        NOBITS       ffffffff835912e0 00002000 000392a8  0 A      0   0  1
[31] .call_sites          NOBITS       ffffffff835ca588 00002000 000e4c58  0 A      0   0  1
[32] .altinstructions     NOBITS       ffffffff836af1e0 00002000 0000cc6c  0 A      0   0  1
[33] .altinstr_replacement NOBITS       ffffffff836bbe4c 00002000 00002517  0 AX     0   0  1
[34] .apicdrivers         NOBITS       ffffffff836be368 00002000 00000028  0 WA     0   0  8
[35] .exit.text           NOBITS       ffffffff836be390 00002000 00002337  0 AX     0   0 16
[36] .smp_locks           NOBITS       ffffffff836c1000 00002000 0000a000  0 A      0   0  4
[37] .data_nosave         NOBITS       ffffffff836cb000 00002000 00001000  0 WA     0   0  4
[38] .bss                 NOBITS       ffffffff836cc000 00002000 01134000  0 WA     0   0 4096
[39] .brk                 NOBITS       ffffffff84800000 00002000 0002c000  0 WA     0   0  1
[40] .init.scratch        NOBITS       ffffffff84a00000 00002000 00400000  0 WA     0   0 32
[41] .debug_aranges       PROGBITS     0000000000000000 00002000 0002a510  0        0   0 16
[42] .debug_info          PROGBITS     0000000000000000 0002c510 0b7e5226  0        0   0  1
[43] .debug_abbrev        PROGBITS     0000000000000000 0b811736 005cea69  0        0   0  1
[44] .debug_line          PROGBITS     0000000000000000 0bde019f 015e216e  0        0   0  1
[45] .debug_frame         PROGBITS     0000000000000000 0d3c2310 002baf10  0        0   0  8
[46] .debug_str           PROGBITS     0000000000000000 0d67d220 003fcaf3  1 MS     0   0  1
[47] .debug_line_str      PROGBITS     0000000000000000 0da79d13 000165dc  1 MS     0   0  1
[48] .debug_loclists      PROGBITS     0000000000000000 0da902ef 01ad700b  0        0   0  1
[49] .debug_rnglists      PROGBITS     0000000000000000 0f5672fa 00377d90  0        0   0  1
[50] .comment             PROGBITS     0000000000000000 0f8df08a 00000036  1 MS     0   0  1
[51] .symtab              SYMTAB       0000000000000000 0f8df0c0 004718d0 24       52 135125  8
[52] .strtab              STRTAB       0000000000000000 0fd50990 003721a4  0        0   0  1
[53] .shstrtab            STRTAB       0000000000000000 100c2b34 0000026a  0        0   0  1

Command we use to split .debug from vmlinux:

eu-strip --strip-all --keep-section=.BTF --remove-comment -f vmlinux.debug vmlinux
k-hagio commented 10 months ago

I could reproduce it, 6.1.12 has .symtab but 6.5.9 doesn't have .symtab after eu-strip --strip-all. It looks like crash can start session at least with --keep-section=.symtab.

gdb-10.2 itself can read a separated .symtab, there may be a way to have crash do the same..

Momoasx commented 8 months ago

Forgot to show run for 8.0.4. It passes further but still fails:

# crash
crash 8.0.4-alt1
...
warning: Loadable section ".notes" outside of ELF segments

crash: cannot determine idle task addresses from init_tasks[] or runqueues[]

crash: cannot resolve "init_task_union"
#

Tail of run with debug:

# crash -d9
...
utsname:
     sysname: Linux
    nodename: REX
     release: 6.5.11-un-def-alt1
     version: #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023
     machine: x86_64
  domainname: (none)
base kernel version: 6.5.11
GETBUF(344 -> 0)
FREEBUF(0)
<readmem: ffffffff821368a0, KVADDR, "accessible check", 8, (ROE|Q), 7ffc4de2c138>
<read_proc_kcore: addr: ffffffff821368a0 paddr: 21368a0 cnt: 8>
<readmem: ffffffff821368a0, KVADDR, "read_string characters", 1499, (ROE|Q), 7ffc4de2cd00>
<read_proc_kcore: addr: ffffffff821368a0 paddr: 21368a0 cnt: 1499>
linux_banner:
Linux version 6.5.11-un-def-alt1 (builder@localhost.localdomain) (gcc-13 (GCC) 13.2.1 20230817 (ALT Sisyphus 13.2.1-alt2), GNU ld (GNU Binutils) 2.41.0.2023082
6) #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023

verify_namelist:
live memory /proc/version:
Linux version 6.5.11-un-def-alt1 (builder@localhost.localdomain) (gcc-13 (GCC) 13.2.1 20230817 (ALT Sisyphus 13.2.1-alt2), GNU ld (GNU Binutils) 2.41.0.20230826) #1 SMP PREEMPT_DYNAMIC Thu Nov  9 02:15:25 UTC 2023
...
ffffffffc21ed080 __func__.56
ffffffffc21ed1fc _note_15
ffffffffc21ed214 _note_14
ffffffffc21ed22c orc_header
ffffffffc21ee000 _MODULE_RODATA_END_xt_conntrack
<readmem: ffffffff84337e40, KVADDR, "nr_threads", 4, (FOE), 55ce4babda1c>
<read_proc_kcore: addr: ffffffff84337e40 paddr: 4337e40 cnt: 4>
crash: per_cpu_symbol_search(per_cpu__runqueues): NULL
crash: cannot determine idle task addresses from init_tasks[] or runqueues[]

crash: cannot resolve "init_task_union"
#