Open zmhu opened 5 months ago
x86架构 最新源代码编译后CPU load高
Mem: 314532K used, 1597772K free, 33540K shrd, 1816K buff, 125520K cached CPU: 1% usr 5% sys 0% nic 92% idle 0% io 0% irq 0% sirq Load average: 0.68 0.58 0.60 2/183 7905 PID PPID USER STAT VSZ %VSZ %CPU COMMAND 29553 2 root RW 0 0% 5% [kworker/2:2+eve] 26019 1 root S 1292m 69% 2% /tmp/etc/passwall2/bin/xray run -c /tmp/etc/passwall2/acl/default/global.jso 32381 5248 root R 1156 0% 0% top
这个5%的system就是 kworker进程, 也不知道内核在忙啥
x86
我抓了个快照
echo l > /proc/sysrq-trigger
结果如下:(几次都得到这样的), CPU-2也不知道在干啥
[36296.642555] NMI backtrace for cpu 1 [36296.646070] CPU: 1 PID: 24626 Comm: bash Not tainted 6.1.83 #0 [36296.651930] Hardware name: LENOVO INVALID/361A, BIOS O1FKT58A 10/21/2016 [36296.658644] Call Trace: [36296.661113] <TASK> [36296.663238] dump_stack_lvl+0x49/0x5f [36296.666940] dump_stack+0x10/0x12 [36296.670291] nmi_cpu_backtrace.cold+0x32/0x7a [36296.674673] ? lapic_can_unplug_cpu+0x80/0x80 [36296.679067] nmi_trigger_cpumask_backtrace+0xe0/0x100 [36296.684149] arch_trigger_cpumask_backtrace+0x14/0x20 [36296.689231] sysrq_handle_showallcpus+0x12/0x20 [36296.693791] __handle_sysrq.cold+0x48/0x120 [36296.698010] write_sysrq_trigger+0x23/0x40 [36296.702138] proc_reg_write+0x58/0x90 [36296.705828] do_iter_write+0xf8/0x1e0 [36296.709522] vfs_writev+0x7b/0x140 [36296.712956] do_writev+0x5b/0xd0 [36296.716221] __x64_sys_writev+0x17/0x20 [36296.720088] do_syscall_64+0x43/0xc0 [36296.723692] entry_SYSCALL_64_after_hwframe+0x63/0xcd [36296.728777] RIP: 0033:0x7f2d224d474e [36296.732379] Code: 89 74 24 10 48 89 e5 48 89 04 24 49 29 c5 48 89 54 24 18 4c 89 6c 24 08 49 01 d5 48 63 7b 78 49 63 d6 4c 89 f8 48 89 ee 0f 05 <48> 89 c7 e8 6a ba fc ff 49 39 c5 75 19 48 8b 43 58 48 8b 53 60 48 [36296.751132] RSP: 002b:00007ffe3e8ba090 EFLAGS: 00000202 ORIG_RAX: 0000000000000014 [36296.758725] RAX: ffffffffffffffda RBX: 00007f2d224ff2c0 RCX: 00007f2d224d474e [36296.765867] RDX: 0000000000000002 RSI: 00007ffe3e8ba090 RDI: 0000000000000001 [36296.773012] RBP: 00007ffe3e8ba090 R08: 0000000000002000 R09: fefefefefefefeff [36296.780158] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001 [36296.787307] R13: 0000000000000002 R14: 0000000000000002 R15: 0000000000000014 [36296.794458] </TASK> [36296.796754] Sending NMI from CPU 1 to CPUs 0,2-3: [36296.801593] NMI backtrace for cpu 0 [36296.801598] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.1.83 #0 [36296.801604] Hardware name: LENOVO INVALID/361A, BIOS O1FKT58A 10/21/2016 [36296.801606] RIP: 0010:mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.801617] Code: 80 cb 01 00 0f 01 c8 48 8b 00 a8 08 75 1c 8b 05 e6 3d fe 00 85 c0 7e 07 0f 00 2d 3b 7a 9b 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 80 cb 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b [36296.801621] RSP: 0018:ffffffff82203dd0 EFLAGS: 00000046 [36296.801626] RAX: 0000000000000064 RBX: ffff888076a2d200 RCX: 0000000000000001 [36296.801630] RDX: 0000000000000000 RSI: ffffffff822d4bc0 RDI: 0000000000000064 [36296.801632] RBP: ffffffff82203dd8 R08: 0000000000000005 R09: 0000000000000101 [36296.801636] R10: 0000000040000000 R11: 736f6d6570736575 R12: 0000000000000005 [36296.801639] R13: ffffffff822d4bc0 R14: ffffffff822d4de0 R15: ffffffff822d4dc8 [36296.801642] FS: 0000000000000000(0000) GS:ffff888076a00000(0000) knlGS:0000000000000000 [36296.801646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [36296.801649] CR2: 00007feb78de22e0 CR3: 000000000520e000 CR4: 00000000001006f0 [36296.801653] Call Trace: [36296.801655] <NMI> [36296.801658] ? show_regs.part.0+0x1e/0x20 [36296.801665] ? show_regs.cold+0x8/0xd [36296.801677] ? nmi_cpu_backtrace.cold+0x1c/0x7a [36296.801681] ? nmi_cpu_backtrace_handler+0xc/0x20 [36296.801687] ? nmi_handle+0x59/0xc0 [36296.801692] ? default_do_nmi+0x47/0x240 [36296.801702] ? exc_nmi+0x103/0x130 [36296.801707] ? end_repeat_nmi+0x16/0x67 [36296.801712] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.801718] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.801723] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.801728] </NMI> [36296.801729] <TASK> [36296.801730] ? intel_idle+0x1c/0x30 [36296.801735] cpuidle_enter_state+0x81/0x2c0 [36296.801743] cpuidle_enter+0x2f/0x40 [36296.801748] call_cpuidle+0x1e/0x40 [36296.801752] do_idle+0x155/0x170 [36296.801756] cpu_startup_entry+0x28/0x30 [36296.801760] rest_init+0xc9/0xd0 [36296.801763] arch_call_rest_init+0x9/0xa [36296.801768] start_kernel+0x672/0x684 [36296.801773] x86_64_start_reservations+0x29/0x2b [36296.801778] x86_64_start_kernel+0x79/0x7f [36296.801782] secondary_startup_64_no_verify+0xce/0xdb [36296.801787] </TASK> [36296.801792] NMI backtrace for cpu 2 [36296.801807] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 6.1.83 #0 [36296.801822] Hardware name: LENOVO INVALID/361A, BIOS O1FKT58A 10/21/2016 [36296.801829] RIP: 0010:mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.801858] Code: 80 cb 01 00 0f 01 c8 48 8b 00 a8 08 75 1c 8b 05 e6 3d fe 00 85 c0 7e 07 0f 00 2d 3b 7a 9b 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 80 cb 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b [36296.801870] RSP: 0018:ffffc900000f7e58 EFLAGS: 00000046 [36296.801884] RAX: 0000000000000064 RBX: ffff888076b2d200 RCX: 0000000000000001 [36296.801893] RDX: 0000000000000000 RSI: ffffffff822d4bc0 RDI: 0000000000000064 [36296.801900] RBP: ffffc900000f7e60 R08: 0000000000000005 R09: 000000000000012e [36296.801908] R10: 0000000000000000 R11: 00000000000000ed R12: 0000000000000005 [36296.801915] R13: ffffffff822d4bc0 R14: ffffffff822d4de0 R15: ffffffff822d4dc8 [36296.801925] FS: 0000000000000000(0000) GS:ffff888076b00000(0000) knlGS:0000000000000000 [36296.801936] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [36296.801944] CR2: 000000000043884b CR3: 000000000520e000 CR4: 00000000001006e0 [36296.801954] Call Trace: [36296.801961] <NMI> [36296.801968] ? show_regs.part.0+0x1e/0x20 [36296.801986] ? show_regs.cold+0x8/0xd [36296.802000] ? nmi_cpu_backtrace.cold+0x1c/0x7a [36296.802015] ? nmi_cpu_backtrace_handler+0xc/0x20 [36296.802032] ? nmi_handle+0x59/0xc0 [36296.802046] ? default_do_nmi+0x47/0x240 [36296.802063] ? exc_nmi+0x103/0x130 [36296.802078] ? end_repeat_nmi+0x16/0x67 [36296.802092] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802108] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802123] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802139] </NMI> [36296.802142] <TASK> [36296.802146] ? intel_idle+0x1c/0x30 [36296.802161] cpuidle_enter_state+0x81/0x2c0 [36296.802181] cpuidle_enter+0x2f/0x40 [36296.802195] call_cpuidle+0x1e/0x40 [36296.802207] do_idle+0x155/0x170 [36296.802218] cpu_startup_entry+0x28/0x30 [36296.802229] start_secondary+0x107/0x110 [36296.802241] secondary_startup_64_no_verify+0xce/0xdb [36296.802256] </TASK> [36296.802261] NMI backtrace for cpu 3 [36296.802273] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.1.83 #0 [36296.802287] Hardware name: LENOVO INVALID/361A, BIOS O1FKT58A 10/21/2016 [36296.802292] RIP: 0010:mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802325] Code: 80 cb 01 00 0f 01 c8 48 8b 00 a8 08 75 1c 8b 05 e6 3d fe 00 85 c0 7e 07 0f 00 2d 3b 7a 9b 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 80 cb 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b [36296.802338] RSP: 0018:ffffc900000ffe58 EFLAGS: 00000046 [36296.802354] RAX: 0000000000000064 RBX: ffff888076bad200 RCX: 0000000000000001 [36296.802364] RDX: 0000000000000000 RSI: ffffffff822d4bc0 RDI: 0000000000000064 [36296.802373] RBP: ffffc900000ffe60 R08: 0000000000000005 R09: 0000000000000000 [36296.802381] R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000005 [36296.802389] R13: ffffffff822d4bc0 R14: ffffffff822d4de0 R15: ffffffff822d4dc8 [36296.802400] FS: 0000000000000000(0000) GS:ffff888076b80000(0000) knlGS:0000000000000000 [36296.802412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [36296.802423] CR2: 00007fff2a10adc8 CR3: 000000000520e000 CR4: 00000000001006e0 [36296.802434] Call Trace: [36296.802440] <NMI> [36296.802447] ? show_regs.part.0+0x1e/0x20 [36296.802465] ? show_regs.cold+0x8/0xd [36296.802481] ? nmi_cpu_backtrace.cold+0x1c/0x7a [36296.802497] ? nmi_cpu_backtrace_handler+0xc/0x20 [36296.802515] ? nmi_handle+0x59/0xc0 [36296.802530] ? default_do_nmi+0x47/0x240 [36296.802548] ? exc_nmi+0x103/0x130 [36296.802565] ? end_repeat_nmi+0x16/0x67 [36296.802581] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802600] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802618] ? mwait_idle_with_hints.constprop.0+0x50/0xa0 [36296.802635] </NMI> [36296.802639] <TASK> [36296.802643] ? intel_idle+0x1c/0x30 [36296.802660] cpuidle_enter_state+0x81/0x2c0 [36296.802680] cpuidle_enter+0x2f/0x40 [36296.802697] call_cpuidle+0x1e/0x40 [36296.802709] do_idle+0x155/0x170 [36296.802722] cpu_startup_entry+0x28/0x30 [36296.802734] start_secondary+0x107/0x110 [36296.802748] secondary_startup_64_no_verify+0xce/0xdb [36296.802764] </TASK>
感觉是内核panic了。换个内核版本试试
我用的7621的CPU也是这样的情况,CPU使用率很高,经常跳到100
6.1好像这样子6.6和5.15正常
660x 7621的内核是5.4和5.10都这样
详细叙述
x86架构 最新源代码编译后CPU load高
这个5%的system就是 kworker进程, 也不知道内核在忙啥
重复 issue
具体型号
x86
详细日志
我抓了个快照
echo l > /proc/sysrq-trigger
结果如下:(几次都得到这样的), CPU-2也不知道在干啥