bengaineyarm
commented
3 years ago Hi
This doesn't look like a gator issue, but a rt kernel issue:
[ 6019.630453@0] kernel BUG at kernel/sched/rt.c:932!
Closed heyg closed 2 years ago
bengaineyarm
commented
3 years ago Hi
This doesn't look like a gator issue, but a rt kernel issue:
[ 6019.630453@0] kernel BUG at kernel/sched/rt.c:932!
heyg
commented
3 years ago Hi Ben, Thanks for your reply.
Actually, I have back to back to 6.5.1 version in upstream/master, it work normal. checkout commit c40528517494ece9c834a3f55c0b215bdbccaec6
I used kernel4.9， I am not sure the kernel needs any other cl?
bengaineyarm
commented
3 years ago Are you saying that all previous version of gator, upto 6.5.1 also trigger this failure, but not 6.5.1 or earlier?
Are you using preempt_rt kernel, or just normal kernel?
heyg
commented
3 years ago Hi Ben, I only try 6.5.1, and kernel4.9 is not enabled CONFIG_PREEMPT_RT.
bengaineyarm
commented
3 years ago Ok, got it.
From that trace, it looks like it is calling into the "CoreOnliner" class in gatord (which is used to try to force all the CPU's to turn on so that gator can identify them), at which point it is triggering a debug warning in the kernel.
[ 6019.617349@0] sched: RT throttling activated for rt_rq ffffffc061b2baa8 (cpu 0) [ 6019.617349@0] potential CPU hogs: [ 6019.617349@0] gatord-cid-0 (3930) [ 6019.625692@0] ------------[ cut here ]------------ [ 6019.630453@0] kernel BUG at kernel/sched/rt.c:932!
A few things you can try:
--disable-cpu-onlining yes option which should disable the CoreOnliner classheyg
commented
3 years ago Hi Ben, Yes, I enabled CONFIG_PANIC_ON_RT_THROTTLING.
In fact, I got useful information in 6.5.1, and my problem has been solved. I am currently stuck in another project and I will follow your suggestions to retry once I have time.
Thank you for your careful analysis.
bengaineyarm
commented
3 years ago :+1: no worries; let me know how you get on, thanks.
base fc2b7126b15caf818f377dae911af0a3c29c65f6, run "./gatord -d", report some err, /data # ./gatord -d [6013.4662145] ERROR: parseCLIArguments(GatorCLIParser.cpp:317): Streamline gatord version 720 (Streamline v7.2) [6013.4684840] INFO: parseXml(xml/PmuXMLParser.cpp:178): Found] lr : [] pstate: 804003c5
[ 6019.730290@0] sp : ffffff80201c7800
[ 6019.733757@0] x29: ffffff80201c7800 x28: ffffffc05efab500
[ 6019.739217@0] x27: ffffff8009b9c9dc x26: ffffffc061b2bab8
[ 6019.744677@0] x25: ffffff8009e65000 x24: ffffff8009d54370
[ 6019.750137@0] x23: ffffff80201c7b30 x22: ffffff80091a70cc
[ 6019.755597@0] x21: 000000000000001d x20: ffffff80201c7a00
[ 6019.761057@0] x19: ffffff800a16e12d x18: ffffffc061b2a950
[ 6019.766517@0] x17: ffffff80201c7b30 x16: ffffff80090db068
[ 6019.771977@0] x15: 0000000000000000 x14: 2e34202020204f20
[ 6019.777437@0] x13: 0000000000000040 x12: 0000000000000000
[ 6019.782897@0] x11: 000000000000000f x10: 00000000000004bd
[ 6019.788357@0] x9 : 0000000000000001 x8 : 0000000000000002
[ 6019.793817@0] x7 : 736b7361745f7472 x6 : ffffff800a16e0cc
[ 6019.799277@0] x5 : 000000000000000a x4 : 0000000000000001
[ 6019.804737@0] x3 : 0000000000000000 x2 : 0000000000000000
[ 6019.810197@0] x1 : 0000000000000001 x0 : ffffff800a149c84
[ 6019.815662@0]
[ 6019.815662@0] X18: 0xffffffc061b2a8d0:
[ 6019.821031@0] a8d0 0000000f 00000000 00000001 00000000 09d591ae ffffff80 0a1704d8 ffffff80
[ 6019.829350@0] a8f0 201c7510 ffffff80 0a1708b8 ffffff80 09d591ae ffffff80 201c7020 ffffff80
[ 6019.837670@0] a910 093eb0c4 ffffff80 61b2a950 ffffffc0 093ea88c ffffff80 004003c5 00000000
[ 6019.845990@0] a930 00000000 00000000 00000000 00000000 ffffffff 0000007f 00000000 00000000
[ 6019.854310@0] a950 00000000 00000000 00000000 00000000 00000016 00000000 00000000 00000000
[ 6019.862630@0] a970 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000007
[ 6019.870950@0] a990 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.879270@0] a9b0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.887592@0]
[ 6019.887592@0] X26: 0xffffffc061b2ba38:
[ 6019.892964@0] ba38 61b2b900 ffffffc0 00000001 00000000 61b2c198 ffffffc0 61b2c198 ffffffc0
[ 6019.901284@0] ba58 0a16bb00 ffffff80 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.909604@0] ba78 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.917924@0] ba98 61b2ba98 ffffffc0 61b2ba98 ffffffc0 00000001 00000000 00000000 00000010
[ 6019.926244@0] bab8 5efab7c0 ffffffc0 5efab7c0 ffffffc0 61b2bac8 ffffffc0 61b2bac8 ffffffc0
[ 6019.934564@0] bad8 61b2bad8 ffffffc0 61b2bad8 ffffffc0 61b2bae8 ffffffc0 61b2bae8 ffffffc0
[ 6019.942884@0] baf8 61b2baf8 ffffffc0 61b2baf8 ffffffc0 61b2bb08 ffffffc0 61b2bb08 ffffffc0
[ 6019.951204@0] bb18 61b2bb18 ffffffc0 61b2bb18 ffffffc0 61b2bb28 ffffffc0 61b2bb28 ffffffc0
[ 6019.959524@0]
[ 6019.959524@0] X28: 0xffffffc05efab480:
[ 6019.964897@0] b480 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.973217@0] b4a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.981537@0] b4c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.989857@0] b4e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6019.998177@0] b500 00000000 00000000 ffffffff ffffffff 00000004 00000000 00000000 00000000
[ 6020.006497@0] b520 201c4000 ffffff80 00000002 00400140 00000000 00000000 00000000 00000000
[ 6020.014817@0] b540 00000001 00000000 00000013 00000000 0015cf70 00000001 001fa7c0 ffffffc0
[ 6020.023137@0] b560 00000000 00000001 00000000 00000065 00000000 00000063 09bb9690 ffffff80
[ 6020.031458@0]
[ 6020.033104@0] ---[ end trace 381ecbcc9c4d2cc5 ]---
[ 6020.037872@0] Call trace:
[ 6020.040472@0] Exception stack(0xffffff80201c7610 to 0xffffff80201c7740)
[ 6020.047059@0] 7600: ffffff800a16e12d 0000007fffffffff
[ 6020.055033@0] 7620: ffffffc061b2a950 ffffff80091f0e2c 00000000804003c5 000000000000003d
[ 6020.063006@0] 7640: ffffff8009e65000 ffffff8009142ce8 ffffff8009d54360 ffffff80201c76a0
[ 6020.070979@0] 7660: ffffff80201c7700 ffffff80201c7700 ffffff80201c76c0 00000000ffffffc8
[ 6020.078952@0] 7680: ffffff80201c7700 ffffff80201c7700 ffffff80201c76c0 00000000ffffffc8
[ 6020.086925@0] 76a0: ffffff80201c7700 ffffff80201c7700 ffffff80201c76c0 00000000ffffffc8
[ 6020.094899@0] 76c0: 000000000000001d ffffff80201c7730 ffffff800a149c84 0000000000000001
[ 6020.102872@0] 76e0: 0000000000000000 0000000000000000 0000000000000001 000000000000000a
[ 6020.110846@0] 7700: ffffff800a16e0cc 736b7361745f7472 0000000000000002 0000000000000001
[ 6020.118818@0] 7720: 00000000000004bd 000000000000000f 0000000000000000 0000000000000040
[ 6020.126793@0] [ffffff80201c7800+ 32][] vmalloc_to_page+0xd8/0xe4
[ 6020.134421@0] [ffffff80201c7820+ 96][] __show_regs+0xac/0x3d0
[ 6020.141789@0] [ffffff80201c7880+ 48][] die+0x98/0x10c
[ 6020.148633@0] [ffffff80201c78b0+ 48][] die+0x64/0xe4
[ 6020.155222@0] [ffffff80201c78e0+ 32][] bug_handler+0x68/0x98
[ 6020.162499@0] [ffffff80201c7900+ 48][] brk_handler+0x94/0xe0
[ 6020.169781@0] [ffffff80201c7930+ 512][] do_debug_exception+0x8c/0x138
[ 6020.177751@0] Exception stack(0xffffff80201c7940 to 0xffffff80201c7a70)
[ 6020.184339@0] 7940: 0000000000000064 0000007fffffffff ffffffc061b2a950 ffffff80091a70cc
[ 6020.192312@0] 7960: 00000000804001c5 000000000000003d ffffffc061b2bab8 0000000000000000
[ 6020.200285@0] 7980: 000000000000006a 00000000000001c0 ffffff80201c79b0 ffffff8009d5ff38
[ 6020.208259@0] 79a0: ffffff8009d5ff50 ffffff8009ff8ef8 ffffff8009fb7010 ffffff8009d5ff80
[ 6020.216232@0] 79c0: 000000010000006a 0000000000000000 ffffff80201c7a70 ffffff80091a72e0
[ 6020.224205@0] 79e0: ffffff8009d5dee8 ffffff80201c7bf3 ffffffc061b2baa8 ffffff80201c7d7c
[ 6020.232179@0] 7a00: 000000000000006a 000000000000006a ffffff8009fb7010 0000000000000005
[ 6020.240152@0] 7a20: 0000004057b65000 0000000000000000 ffffff800a16e12d 6231363063666666
[ 6020.248125@0] 7a40: 0000000000000006 0000000000000001 00000000000004b4 0000000000000008
[ 6020.256098@0] 7a60: 0000000000000000 0000000000000080
[ 6020.261126@0] [ffffff80201c7b30+ 592][] el1_dbg+0x44/0xa0
[ 6020.268062@0] [ffffff80201c7d80+ 48][] update_curr_rt+0x0/0x1a4
[ 6020.275600@0] [ffffff80201c7db0+ 48][] update_curr_rt+0x134/0x1a4
[ 6020.283313@0] [ffffff80201c7de0+ 48][] pick_next_task_rt+0xa0/0xbc
[ 6020.291115@0] [ffffff80201c7e10+ 112][] __schedule+0x430/0x718
[ 6020.298479@0] [ffffff80201c7e80+ 32][] schedule+0x3c/0x9c
[ 6020.305500@0] [ffffff80201c7ea0+ 0][] sys_sched_yield+0x6c/0x8c
[ 6020.313126@0] [0000000000000000+ 0][] el0_svc_naked+0x34/0x38
[ 6020.320579@0] pc : [] lr : [] pstate: 804001c5
[ 6020.328117@0] sp : ffffff80201c7b30
[ 6020.331585@0] x29: ffffff80201c7b30 x28: ffffffc05efab500
[ 6020.337044@0] x27: ffffff8009b9c9dc x26: ffffffc061b2bab8
[ 6020.342504@0] x25: ffffffc061b2bab8 x24: ffffff80201c7d7c
[ 6020.347964@0] x23: ffffff8009d5df08 x22: ffffff80201c7d7c
[ 6020.353424@0] x21: ffffffc061b2baa8 x20: ffffff80201c7bf3
[ 6020.358884@0] x19: 0000000000000064 x18: 0000007fb5d6c988
[ 6020.364344@0] x17: 0000007fb5cd8c80 x16: ffffff80090db068
[ 6020.369804@0] x15: 0000000000000000 x14: 65746f700a293020
[ 6020.375264@0] x13: 0000000000000080 x12: 0000000000000000
[ 6020.380724@0] x11: 0000000000000008 x10: 00000000000004b4
[ 6020.386184@0] x9 : 0000000000000001 x8 : 0000000000000006
[ 6020.391644@0] x7 : 6231363063666666 x6 : ffffff800a16e12d
[ 6020.397104@0] x5 : 0000000000000000 x4 : 0000004057b65000
[ 6020.402564@0] x3 : 0000000000000005 x2 : ffffff8009fb7010
[ 6020.408024@0] x1 : 000000000000006a x0 : 000000000000006a
[ 6020.413488@0]
[ 6020.413488@0] X21: 0xffffffc061b2ba28:
[ 6020.418858@0] ba28 0015d144 00000001 00000000 00000000 61b2b900 ffffffc0 00000001 00000000
[ 6020.427177@0] ba48 61b2c198 ffffffc0 61b2c198 ffffffc0 0a16bb00 ffffff80 00000000 00000000
[ 6020.435497@0] ba68 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.443817@0] ba88 00000000 00000000 00000000 00000000 61b2ba98 ffffffc0 61b2ba98 ffffffc0
[ 6020.452137@0] baa8 00000001 00000000 00000000 00000010 5efab7c0 ffffffc0 5efab7c0 ffffffc0
[ 6020.460457@0] bac8 61b2bac8 ffffffc0 61b2bac8 ffffffc0 61b2bad8 ffffffc0 61b2bad8 ffffffc0
[ 6020.468777@0] bae8 61b2bae8 ffffffc0 61b2bae8 ffffffc0 61b2baf8 ffffffc0 61b2baf8 ffffffc0
[ 6020.477097@0] bb08 61b2bb08 ffffffc0 61b2bb08 ffffffc0 61b2bb18 ffffffc0 61b2bb18 ffffffc0
[ 6020.485418@0]
[ 6020.485418@0] X25: 0xffffffc061b2ba38:
[ 6020.490790@0] ba38 61b2b900 ffffffc0 00000001 00000000 61b2c198 ffffffc0 61b2c198 ffffffc0
[ 6020.499110@0] ba58 0a16bb00 ffffff80 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.507430@0] ba78 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.515750@0] ba98 61b2ba98 ffffffc0 61b2ba98 ffffffc0 00000001 00000000 00000000 00000010
[ 6020.524070@0] bab8 5efab7c0 ffffffc0 5efab7c0 ffffffc0 61b2bac8 ffffffc0 61b2bac8 ffffffc0
[ 6020.532390@0] bad8 61b2bad8 ffffffc0 61b2bad8 ffffffc0 61b2bae8 ffffffc0 61b2bae8 ffffffc0
[ 6020.540710@0] baf8 61b2baf8 ffffffc0 61b2baf8 ffffffc0 61b2bb08 ffffffc0 61b2bb08 ffffffc0
[ 6020.549030@0] bb18 61b2bb18 ffffffc0 61b2bb18 ffffffc0 61b2bb28 ffffffc0 61b2bb28 ffffffc0
[ 6020.557351@0]
[ 6020.557351@0] X26: 0xffffffc061b2ba38:
[ 6020.562724@0] ba38 61b2b900 ffffffc0 00000001 00000000 61b2c198 ffffffc0 61b2c198 ffffffc0
[ 6020.571044@0] ba58 0a16bb00 ffffff80 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.579363@0] ba78 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.587684@0] ba98 61b2ba98 ffffffc0 61b2ba98 ffffffc0 00000001 00000000 00000000 00000010
[ 6020.596004@0] bab8 5efab7c0 ffffffc0 5efab7c0 ffffffc0 61b2bac8 ffffffc0 61b2bac8 ffffffc0
[ 6020.604323@0] bad8 61b2bad8 ffffffc0 61b2bad8 ffffffc0 61b2bae8 ffffffc0 61b2bae8 ffffffc0
[ 6020.612644@0] baf8 61b2baf8 ffffffc0 61b2baf8 ffffffc0 61b2bb08 ffffffc0 61b2bb08 ffffffc0
[ 6020.620963@0] bb18 61b2bb18 ffffffc0 61b2bb18 ffffffc0 61b2bb28 ffffffc0 61b2bb28 ffffffc0
[ 6020.629284@0]
[ 6020.629284@0] X28: 0xffffffc05efab480:
[ 6020.634657@0] b480 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.642977@0] b4a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.651297@0] b4c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.659617@0] b4e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 6020.667937@0] b500 00000000 00000000 ffffffff ffffffff 00000004 00000000 00000000 00000000
[ 6020.676257@0] b520 201c4000 ffffff80 00000002 00400140 00000000 00000000 00000000 00000000
[ 6020.684577@0] b540 00000001 00000000 00000013 00000000 0015cf70 00000001 001fa7c0 ffffffc0
[ 6020.692897@0] b560 00000000 00000001 00000000 00000065 00000000 00000063 09bb9690 ffffff80
[ 6020.701217@0]
[ 6020.702865@0] reg value pfn reg value pfn
[ 6020.710233@0] r0 : 000000000000006a -------- r1 : 000000000000006a --------
[ 6020.717599@0] r2 : ffffff8009fb7010 0 r3 : 0000000000000005 --------
[ 6020.724965@0] r4 : 0000004057b65000 -------- r5 : 0000000000000000 --------
[ 6020.732331@0] r6 : ffffff800a16e12d -------- r7 : 6231363063666666 --------
[ 6020.739697@0] r8 : 0000000000000006 -------- r9 : 0000000000000001 --------
[ 6020.747064@0] r10: 00000000000004b4 -------- r11: 0000000000000008 --------
[ 6020.754431@0] r12: 0000000000000000 -------- r13: 0000000000000080 --------
[ 6020.761798@0] r14: 65746f700a293020 -------- r15: 0000000000000000 --------
[ 6020.769164@0] r16: ffffff80090db068 10db r17: 0000007fb5cd8c80 5acf3
[ 6020.776532@0] r18: 0000007fb5d6c988 23ee r19: 0000000000000064 --------
[ 6020.783898@0] r20: ffffff80201c7bf3 50b50 r21: ffffffc061b2baa8 --------
[ 6020.791264@0] r22: ffffff80201c7d7c 50b50 r23: ffffff8009d5df08 --------
[ 6020.798631@0] r24: ffffff80201c7d7c 50b50 r25: ffffffc061b2bab8 --------
[ 6020.805997@0] r26: ffffffc061b2bab8 -------- r27: ffffff8009b9c9dc 1b9c
[ 6020.813364@0] r28: ffffffc05efab500 -------- r29: ffffff80201c7b30 50b50
[ 6020.820732@0] r30: ffffff80091a70cc 11a7] dump_throttled_rt_tasks+0x70/0x134
[ 6021.257534@0] [ffffff80201c7d80+ 48][] update_curr_rt+0x0/0x1a4
[ 6021.265073@0] [ffffff80201c7db0+ 48][] update_curr_rt+0x134/0x1a4
[ 6021.272786@0] [ffffff80201c7de0+ 48][] pick_next_task_rt+0xa0/0xbc
[ 6021.280588@0] [ffffff80201c7e10+ 112][] __schedule+0x430/0x718
[ 6021.287952@0] [ffffff80201c7e80+ 32][] schedule+0x3c/0x9c
[ 6021.294972@0] [ffffff80201c7ea0+ 0][] sys_sched_yield+0x6c/0x8c
[ 6021.302600@0] [0000000000000000+ 0][] el0_svc_naked+0x34/0x38
[ 6021.310053@0] Code: d0005da0 910163a1 913ba000 9400005f (d4210000)
[ 6020.825325@0] pc : ffffff80091a70cc 11a7 [ 6020.829745@0] sp : ffffff80201c7b30 50b50 [ 6020.834164@0] unused : 0000000000000000 -------- [ 6020.838932@0] Process gatord-cid-0 (pid: 3930, stack limit = 0xffffff80201c4000) [ 6020.846298@0] Stack: (0xffffff80201c7b30 to 0xffffff80201c8000) [ 6020.852192@0] 7b20: ffffff80201c7d80 ffffff80090f5590 [ 6020.860167@0] 7b40: ffffffc061b2baa8 0000000000000000 ffffff800a16c5e0 ffffffc061b2c148 [ 6020.868139@0] 7b60: 0000000000000000 ffffff8009ffa000 ffffffc05efabb38 ffffffc061b2b900 [ 6020.876112@0] 7b80: ffffffc05047b000 52203a6465686373 74746f7268742054 74636120676e696c [ 6020.884085@0] 7ba0: 6620646574617669 71725f747220726f 6366666666666620 6161623262313630 [ 6020.892059@0] 7bc0: 3020757063282038 746e65746f700a29 20555043206c6169 67090a3a73676f68 [ 6020.900032@0] 7be0: 69632d64726f7461 3339332820302d64 ffffffc0000a2930 ffffffc061b20768 [ 6020.908006@0] 7c00: ffffffc05dc52b40 ffffff8009bec9d8 0000000000001000 ffffffc05dc52ac0 [ 6020.915979@0] 7c20: 0000007fb0001e20 ffffff80201c7de0 ffffff80201c7c70 ffffff80093fd4b8 [ 6020.923952@0] 7c40: ffffff80201c7d10 ffffff80201c7d50 ffffff80201c7c80 ffffff800908b2a0 [ 6020.931925@0] 7c60: ffffff8009fb7000 ffffff80201c7cd0 ffffff80201c7c80 0000000000000000 [ 6020.939899@0] 7c80: ffffff80201c7cc0 ffffff800908b434 ffffff80201c7d10 ffffff80201c7d00 [ 6020.947872@0] 7ca0: ffffffc05efab500 0000000000000002 0000000000000000 0000000000000002 [ 6020.955845@0] 7cc0: 0000000000007617 ffffffc05efab500 0000000000000000 0000000000000002 [ 6020.963819@0] 7ce0: ffffff80201c7d00 ffffff8009ba1450 ffffff800a16c5e0 0000000000000003 [ 6020.971792@0] 7d00: ffffff80201c7d20 ffffff80090f3728 ffffffc061b79aa8 0000000000000001 [ 6020.979765@0] 7d20: ffffff80201c7d80 ffffff80090f5528 ffffffc061b2baa8 0000000000000000 [ 6020.987739@0] 7d40: ffffff80201c7d60 ffffff8009ba0ea8 ffffffc061b2c148 0000000000000000 [ 6020.995712@0] 7d60: ffffff80201c7d80 ffffff80090f5530 ffffffc061b2baa8 ffffffc05efab500 [ 6021.003687@0] 7d80: ffffff80201c7db0 ffffff80090f56c4 ffffffc061b2c148 ffffffc061b2baa8 [ 6021.011659@0] 7da0: ffffffc061b2b900 0000000000000465 ffffff80201c7de0 ffffff80090f58d8 [ 6021.019632@0] 7dc0: ffffffc061b2b900 ffffff8009bb9690 ffffffc061b2b900 ffffffc05efab500 [ 6021.027606@0] 7de0: ffffff80201c7e10 ffffff8009b9c6b8 ffffff8009bb9000 ffffff8009bb9810 [ 6021.035579@0] 7e00: ffffff8009fc6000 ffffffc05efab500 ffffff80201c7e80 ffffff8009b9c9dc [ 6021.043552@0] 7e20: ffffffc05efab500 0000004057b65000 ffffffffffffffff 0000007fb5cd8c88 [ 6021.051525@0] 7e40: 0000000000000000 0000000000000015 0000000000000123 000000000000007c [ 6021.059499@0] 7e60: ffffff8009bb2000 ffffffc05efab500 ffffffc05efab500 0000004057b65000 [ 6021.067473@0] 7e80: ffffff80201c7ea0 ffffff80090db0d4 ffffffc061b2b900 ffffff80090db084 [ 6021.075445@0] 7ea0: 0000000000000000 ffffff8009083900 ffffffffffffff01 0000000000000000 [ 6021.083419@0] 7ec0: 0000000000000000 0000000000000000 0000007fb0000df0 0000007fb00008d0 [ 6021.091392@0] 7ee0: 0000000000000003 0000000000000000 0000000000000001 0000000000000000 [ 6021.099365@0] 7f00: 000000000000007c 0000007fa8000f40 0000000000000001 0000000000000001 [ 6021.107339@0] 7f20: 0000000000000000 0000000000000000 0000007fb00008d0 0000000000000000 [ 6021.115312@0] 7f40: 000000557a783db8 0000007fb5cd8c80 0000007fb5d6c988 00000055ae80ed30 [ 6021.123285@0] 7f60: 0000000000000000 0000007fb5ac7848 0000007fb0000df8 0000007fb0000d7c [ 6021.131259@0] 7f80: 0000007fb5ac7878 0000000000000000 0000000000000000 0000000000000000 [ 6021.139232@0] 7fa0: 0000007fb5ac7878 0000007fb5ac7990 000000557a5d9340 0000007fb5ac7840 [ 6021.147205@0] 7fc0: 0000007fb5cd8c88 0000000000000000 0000000000000000 000000000000007c [ 6021.155178@0] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 6021.163151@0] Call trace: [ 6021.165752@0] Exception stack(0xffffff80201c7940 to 0xffffff80201c7a70) [ 6021.172339@0] 7940: 0000000000000064 0000007fffffffff ffffffc061b2a950 ffffff80091a70cc [ 6021.180312@0] 7960: 00000000804001c5 000000000000003d ffffffc061b2bab8 0000000000000000 [ 6021.188285@0] 7980: 000000000000006a 00000000000001c0 ffffff80201c79b0 ffffff8009d5ff38 [ 6021.196259@0] 79a0: ffffff8009d5ff50 ffffff8009ff8ef8 ffffff8009fb7010 ffffff8009d5ff80 [ 6021.204232@0] 79c0: 000000010000006a 0000000000000000 ffffff80201c7a70 ffffff80091a72e0 [ 6021.212205@0] 79e0: ffffff8009d5dee8 ffffff80201c7bf3 ffffffc061b2baa8 ffffff80201c7d7c [ 6021.220179@0] 7a00: 000000000000006a 000000000000006a ffffff8009fb7010 0000000000000005 [ 6021.228152@0] 7a20: 0000004057b65000 0000000000000000 ffffff800a16e12d 6231363063666666 [ 6021.236125@0] 7a40: 0000000000000006 0000000000000001 00000000000004b4 0000000000000008 [ 6021.244098@0] 7a60: 0000000000000000 0000000000000080 [ 6021.249128@0] [ffffff80201c7b30+ 592][