microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.47k stars 822 forks source link

Kernel Panic Not Syncing Disk & Segmentation Fault #12105

Closed billpull closed 1 month ago

billpull commented 1 month ago

Windows Version

Microsoft Windows [Version 10.0.22631.4169]

WSL Version

2.2.4.0

Are you using WSL 1 or WSL 2?

Kernel Version

5.15.153.1-2

Distro Version

Ubuntu 24.04

Other Software

Docker Desktop v4.34.2 VS Code 1.93.1

Repro Steps

Running Docker or just running commands in WSL after some amount of time will cause a segmentation fault or the entire system to crash.

Some Examples

Expected Behavior

Can use WSL without crashing constantly

Actual Behavior

Segmentation fault & constant crashing

Diagnostic Logs

Event Viewer Log (Hyper-V-Worker)

'DF1A0FF0-C675-432E-88E2-909B39E38943' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x0, ErrorCode1: 0x0, ErrorCode2: 0x0, ErrorCode3: 0x0, ErrorCode4: 0x0.  PreOSId: 0.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID DF1A0FF0-C675-432E-88E2-909B39E38943)

Guest message:
[17015.052604] PGD 100000067 P4D 100000067 PUD 100225067 PMD 12c071067 PTE 0
[17015.053127] Oops: 0000 [#1] SMP NOPTI
[17015.053267] CPU: 23 PID: 2092 Comm: 1_aux Tainted: G        W         5.15.153.1-microsoft-standard-WSL2 #1
[17015.053617] RIP: 0010:__list_del_entry_valid+0x3a/0xe0
[17015.053846] Code: 85 c0 74 4e 48 b8 00 01 00 00 00 00 ad de 48 39 c2 74 54 48 b8 22 01 00 00 00 00 ad de 49 39 c0 74 5a 4d 8b 00 49 39 f0 75 6a <48> 8b 52 08 4c 39 c2 75 79 b8 01 00 00 00 c3 cc cc cc cc 48 c7 c7
[17015.054517] RSP: 0018:ffffaa25ca403c10 EFLAGS: 00010046
[17015.054678] RAX: dead000000000122 RBX: 0000000000000000 RCX: 00000000001c0bef
[17015.054919] RDX: ffffaa25c78a7ca0 RSI: fffff5024702fbc8 RDI: fffff5024702fbc8
[17015.055163] RBP: 00000000001c0bee R08: fffff5024702fbc8 R09: 0000000000000009
[17015.055403] R10: 0000000003d8c858 R11: 0000000003d8c8a8 R12: ffffffffb9029f00
[17015.055662] R13: 00000000001c0bef R14: 0000000000000000 R15: fffff5024702fb80
[17015.056085] FS:  00007fdf2082fb38(0000) GS:ffff95165fbc0000(0000) knlGS:0000000000000000
[17015.056599] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17015.056805] CR2: ffffaa25c78a7ca8 CR3: 000000017383c000 CR4: 0000000000350ea0
[17015.057099] Call Trace:
[17015.057194]  <TASK>
[17015.057297]  ? __die_body.cold+0x1a/0x1f
[17015.057441]  ? page_fault_oops+0xae/0x250
[17015.057568]  ? exc_page_fault+0x86/0x100
[17015.057703]  ? asm_exc_page_fault+0x22/0x30
[17015.057866]  ? __list_del_entry_valid+0x3a/0xe0
[17015.058051]  __free_one_page+0x217/0x430
[17015.058171]  free_pcppages_bulk+0x292/0x360
[17015.058272]  free_unref_page_list+0x187/0x280
[17015.058445]  release_pages+0x182/0x4b0
[17015.058582]  tlb_finish_mmu+0x7a/0x1a0
[17015.058711]  unmap_region+0xf8/0x130
[17015.058834]  __do_munmap+0x1eb/0x510
[17015.058956]  __vm_munmap+0x75/0x120
[17015.059077]  __x64_sys_munmap+0x17/0x20
[17015.059199]  do_syscall_64+0x38/0xc0
[17015.059322]  entry_SYSCALL_64_after_hwframe+0x62/0xcc
[17015.059485] RIP: 0033:0x7fdf699b35cd
[17015.059607] Code: 00 00 00 0f 05 48 89 c7 e8 6c 87 fe ff 5a c3 55 48 89 f5 53 48 89 fb 50 e8 ec ed 01 00 b8 0b 00 00 00 48 89 df 48 89 ee 0f 05 <48> 89 c7 e8 47 87 fe ff 5a 5b 5d c3 31 c0 83 fa 04 74 0c 48 63 d2
[17015.060202] RSP: 002b:00007fdf2082f900 EFLAGS: 00000246 ORIG_RAX: 000000000000000b
[17015.060449] RAX: ffffffffffffffda RBX: 00007fdf1f27c000 RCX: 00007fdf699b35cd
[17015.060695] RDX: 0000000000000000 RSI: 0000000000180000 RDI: 00007fdf1f27c000
[17015.060945] RBP: 0000000000180000 R08: 0000000000000000 R09: 0000000000002018
[17015.061184] R10: 0000000000000004 R11: 0000000000000246 R12: 00007fdf6970a340
[17015.061442] R13: 00007fdf26c9dab0 R14: 00007fdf26c9da80 R15: 0000000000000000
[17015.061709]  </TASK>
[17015.061799] Modules linked in:
[17015.061921] CR2: ffffaa25c78a7ca8
[17015.062043] ---[ end trace c21199b25c59b53c ]---
[17015.062206] RIP: 0010:__list_del_entry_valid+0x3a/0xe0
[17015.062367] Code: 85 c0 74 4e 48 b8 00 01 00 00 00 00 ad de 48 39 c2 74 54 48 b8 22 01 00 00 00 00 ad de 49 39 c0 74 5a 4d 8b 00 49 39 f0 75 6a <48> 8b 52 08 4c 39 c2 75 79 b8 01 00 00 00 c3 cc cc cc cc 48 c7 c7
[17015.062936] RSP: 0018:ffffaa25ca403c10 EFLAGS: 00010046
[17015.063098] RAX: dead000000000122 RBX: 0000000000000000 RCX: 00000000001c0bef
[17015.063348] RDX: ffffaa25c78a7ca0 RSI: fffff5024702fbc8 RDI: fffff5024702fbc8
[17015.063649] RBP: 00000000001c0bee R08: fffff5024702fbc8 R09: 0000000000000009
[17015.063904] R10: 0000000003d8c858 R11: 0000000003d8c8a8 R12: ffffffffb9029f00
[17015.064146] R13: 00000000001c0bef R14: 0000000000000000 R15: fffff5024702fb80
[17015.064402] FS:  00007fdf2082fb38(0000) GS:ffff95165fbc0000(0000) knlGS:0000000000000000
[17015.064647] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17015.064911] CR2: ffffaa25c78a7ca8 CR3: 000000017383c000 CR4: 0000000000350ea0
[17015.065242] Kernel panic - not syncing: Fatal exception
[17016.118500] Shutting down cpus with NMI
[17016.138840] Kernel Offset: 0x36000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

Additional Event Viewer Critical Log

'E2E5EA33-0A83-4D11-AE0D-5936EBD45BE3' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x0, ErrorCode1: 0x0, ErrorCode2: 0x0, ErrorCode3: 0x0, ErrorCode4: 0x0.  PreOSId: 0.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID E2E5EA33-0A83-4D11-AE0D-5936EBD45BE3)

Guest message:
[ 2910.839912] systemd-journald[55]: Time jumped backwards, rotating.
[ 2938.926088] systemd-journald[55]: Time jumped backwards, rotating.
[ 2967.041666] systemd-journald[55]: Time jumped backwards, rotating.
[ 2982.364941] divide error: 0000 [#1] SMP NOPTI
[ 2982.365331] CPU: 3 PID: 44492 Comm: gcc Not tainted 5.15.153.1-microsoft-standard-WSL2 #1
[ 2982.365669] RIP: 0010:__check_heap_object+0x39/0xd0
[ 2982.365955] Code: 89 cf 48 2b 15 58 ce 43 01 48 c1 fa 06 48 c1 e2 0c 48 03 15 59 ce 43 01 48 39 d0 0f 82 83 00 00 00 41 8b 71 18 48 29 d0 48 99 <48> f7 fe 41 8b 81 b0 00 00 00 39 d0 77 1b 41 8b 89 b4 00 00 00 41
[ 2982.367078] RSP: 0018:ffffac7a15607e28 EFLAGS: 00010206
[ 2982.367359] RAX: 0000000000000c36 RBX: 0000000000000003 RCX: 0000000000000000
[ 2982.367774] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 2982.368226] RBP: ffff90013d8c8c39 R08: 0000000000000003 R09: ffff8fffe514f2d9
[ 2982.368637] R10: fefefefefefefeff R11: 8080808080808080 R12: ffff90013d8c8c36
[ 2982.369081] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 2982.369471] FS:  00007f8ee52e3740(0000) GS:ffff90069f6c0000(0000) knlGS:0000000000000000
[ 2982.369908] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2982.370233] CR2: 00007f42fa8dc024 CR3: 00000001b0278000 CR4: 0000000000350ea0
[ 2982.370662] Call Trace:
[ 2982.370815]  <TASK>
[ 2982.370943]  ? __die_body.cold+0x1a/0x1f
[ 2982.371122]  ? die+0x2b/0x50
[ 2982.371297]  ? do_trap+0xd0/0x120
[ 2982.371474]  ? do_error_trap+0x65/0x80
[ 2982.371930]  ? __check_heap_object+0x39/0xd0
[ 2982.372323]  ? exc_divide_error+0x36/0x50
[ 2982.372496]  ? __check_heap_object+0x39/0xd0
[ 2982.372694]  ? asm_exc_divide_error+0x16/0x20
[ 2982.372930]  ? __check_heap_object+0x39/0xd0
[ 2982.373345]  __check_object_size+0x146/0x160
[ 2982.373619]  copy_strings.isra.0+0x17b/0x370
[ 2982.373902]  do_execveat_common.isra.0+0x16b/0x230
[ 2982.374154]  __x64_sys_execve+0x33/0x40
[ 2982.374369]  do_syscall_64+0x38/0xc0
[ 2982.374651]  entry_SYSCALL_64_after_hwframe+0x62/0xcc
[ 2982.374917] RIP: 0033:0x7f8ee53d4f3b
[ 2982.375120] Code: 0f 1e fa 48 8b 05 85 40 11 00 48 8b 10 e9 0d 00 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 f3 0f 1e fa b8 3b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ad 3e 11 00 f7 d8 64 89 01 48
[ 2982.375901] RSP: 002b:00007fff6ac160b8 EFLAGS: 00000202 ORIG_RAX: 000000000000003b
[ 2982.376199] RAX: ffffffffffffffda RBX: 00007fff6ac1740e RCX: 00007f8ee53d4f3b
[ 2982.376527] RDX: 0000000000b61230 RSI: 0000000000b62218 RDI: 00007fff6ac160c0
[ 2982.376915] RBP: 00007fff6ac164d0 R08: 0000000000000fff R09: 0000000000000000
[ 2982.377280] R10: 00007f8ee52f8ec8 R11: 0000000000000202 R12: 0000000000b5ce90
[ 2982.377606] R13: 00007fff6ac173f3 R14: 0000000000000003 R15: 0000000000b62218
[ 2982.377924]  </TASK>
[ 2982.378032] Modules linked in:
[ 2982.378237] ---[ end trace b9440db163de106a ]---
[ 2982.378471] RIP: 0010:__check_heap_object+0x39/0xd0
[ 2982.378677] Code: 89 cf 48 2b 15 58 ce 43 01 48 c1 fa 06 48 c1 e2 0c 48 03 15 59 ce 43 01 48 39 d0 0f 82 83 00 00 00 41 8b 71 18 48 29 d0 48 99 <48> f7 fe 41 8b 81 b0 00 00 00 39 d0 77 1b 41 8b 89 b4 00 00 00 41
[ 2982.379650] RSP: 0018:ffffac7a15607e28 EFLAGS: 00010206
[ 2982.379878] RAX: 0000000000000c36 RBX: 0000000000000003 RCX: 0000000000000000
[ 2982.380266] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 2982.380555] RBP: ffff90013d8c8c39 R08: 0000000000000003 R09: ffff8fffe514f2d9
[ 2982.380889] R10: fefefefefefefeff R11: 8080808080808080 R12: ffff90013d8c8c36
[ 2982.381272] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
[ 2982.381648] FS:  00007f8ee52e3740(0000) GS:ffff90069f6c0000(0000) knlGS:0000000000000000
[ 2982.381951] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2982.382255] CR2: 00007f42fa8dc024 CR3: 00000001b0278000 CR4: 0000000000350ea0
[ 2982.382559] Kernel panic - not syncing: Fatal exception
[ 2982.402882] Kernel Offset: 0xf000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

Exception from Microsoft.Windows.Plan9Server

VhdmpiMainOffloadWriteCompletion: FSCTL_OFFLOAD_WRITE failure status: 0xc00000bb
github-actions[bot] commented 1 month ago

Logs are required for review from WSL team

If this a feature request, please reply with '/feature'. If this is a question, reply with '/question'. Otherwise please attach logs by following the instructions below, your issue will not be reviewed unless they are added. These logs will help us understand what is going on in your machine.

How to collect WSL logs Download and execute [collect-wsl-logs.ps1](https://github.com/Microsoft/WSL/blob/master/diagnostics/collect-wsl-logs.ps1) in an **administrative powershell prompt**: ``` Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1 Set-ExecutionPolicy Bypass -Scope Process -Force .\collect-wsl-logs.ps1 ``` The script will output the path of the log file once done. If this is a networking issue, please use [collect-networking-logs.ps1](https://github.com/Microsoft/WSL/blob/master/diagnostics/collect-networking-logs.ps1), following the instructions [here](https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#collect-wsl-logs-for-networking-issues) Once completed please upload the output files to this Github issue. [Click here for more info on logging](https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#8-collect-wsl-logs-recommended-method) If you choose to email these logs instead of attaching to the bug, please send them to wsl-gh-logs@microsoft.com with the number of the github issue in the subject, and in the message a link to your comment in the github issue and reply with '/emailed-logs'.

View similar issues

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it!

Open similar issues:

Closed similar issues:

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

billpull commented 1 month ago

/emailed-logs

github-actions[bot] commented 1 month ago
Diagnostic information ``` Found '/emailed-logs', adding tag 'emailed-logs' ```
OneBlue commented 1 month ago

Thank you @billpull. Just to check that this is not a hardware issue, can you run a memory check and validate that it comes out without errors ?

R-Bower commented 2 weeks ago

@billpull I'm seeing something similar on my system. Did you ever resolve this?