Open michaeldexter opened 1 year ago
Update. scrub reports no errors. Removing all files reduces the capacity to 44% used. Explorer becomes unresponsive. Panic. Dump analysis to come.
The datestamp on the dump appears to be wrong. My apologies if this is from an earlier crash:
Unable to load image \SystemRoot\system32\DRIVERS\OpenZFS.sys, Win32 error 0n2 KEY_VALUES_STRING: 1 Key : Analysis.CPU.mSec Value: 3014 Key : Analysis.DebugAnalysisManager Value: Create Key : Analysis.Elapsed.mSec Value: 10296 Key : Analysis.Init.CPU.mSec Value: 7311 Key : Analysis.Init.Elapsed.mSec Value: 67257 Key : Analysis.Memory.CommitPeak.Mb Value: 90 Key : WER.OS.Branch Value: rs5_release Key : WER.OS.Timestamp Value: 2018-09-14T14:34:00Z Key : WER.OS.Version Value: 10.0.17763.1 BUGCHECK_CODE: 3b BUGCHECK_P1: c0000005 BUGCHECK_P2: fffff80015fba54c BUGCHECK_P3: ffffdf044b882430 BUGCHECK_P4: 0
CONTEXT: ffffdf044b882430 -- (.cxr 0xffffdf044b882430)
rax=0000000000000000 rbx=ffff8a09bd19edb0 rcx=ffff8a09be57fa10
rdx=0000000000000000 rsi=0000000000000000 rdi=ffff8a09c06cadc0
rip=fffff80015fba54c rsp=ffffdf044b882e20 rbp=ffffdf044b8837e0
r8=0000000000000001 r9=0000000000000001 r10=fffff800161f4ad9
r11=3433323164636261 r12=ffff8a097a851580 r13=ffffdf044b883f00
r14=ffff8a09c090e7f0 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00010282
OpenZFS+0x30a54c:
fffff80015fba54c 488b4010 mov rax,qword ptr [rax+10h] ds:002b:00000000
00000010=????????????????
Resetting default scope
BLACKBOXBSD: 1 (!blackboxbsd)
BLACKBOXPNP: 1 (!blackboxpnp)
PROCESS_NAME: AJA System Test.exe
STACK_TEXT:
ffffdf044b882e20 fffff800
161b9cc6 : ffff8a0981dd6940 00000000
00000002 0000000000000000 00000000
00000008 : OpenZFS+0x30a54c
ffffdf044b882e70 fffff800
161b9c32 : ffffdf044b882f70 ffff8a09
c090e7f0 0000000000001000 00000000
00000000 : OpenZFS+0x509cc6
ffffdf044b882eb0 fffff800
15e718e5 : 0000000000000002 00000000
00000000 deadbeefdeadbeef ffff8a09
be970080 : OpenZFS+0x509c32
ffffdf044b882fc0 fffff800
15e6fd3b : 0000000000000000 00000000
00000001 ffffcf01964ce180 fffff800
0b8378e7 : OpenZFS+0x1c18e5
ffffdf044b883180 fffff800
15e6bec5 : ffff8a0982463f50 ffff8a09
8164fe00 0000000000000000 00000000
00000000 : OpenZFS+0x1bfd3b
ffffdf044b8832f0 fffff800
15e6bb9d : 0000000000000000 ffffdf04
4b8837e0 0000000000000015 ffff8a09
8000a040 : OpenZFS+0x1bbec5
ffffdf044b883330 fffff800
15e6baff : ffffdf044b8833f0 ffffdf04
4b8833f0 ffffdf044b8833e0 fffff800
15f0cae7 : OpenZFS+0x1bbb9d
ffffdf044b883380 fffff800
161b8c40 : 3930413846464646 30383030
37394542 0000000000000001 ffff8a09
c090e7f0 : OpenZFS+0x1bbaff
ffffdf044b883410 fffff800
15fab5b7 : 0000000000000000 ffff8a09
c090e7f0 ffffdf044b883f00 00000000
00000000 : OpenZFS+0x508c40
ffffdf044b883760 fffff800
15fb0e0f : fffff80017560d80 fffff800
15ccb2c8 0000000000000000 00000000
00000000 : OpenZFS+0x2fb5b7
ffffdf044b8838e0 fffff800
15fad432 : 0000000000000000 00000000
000002c0 0000000000000030 ffff8a09
7a8a68a0 : OpenZFS+0x300e0f
ffffdf044b883a50 fffff800
0b82e209 : ffffdf044b87f000 ffffdf04
4b883c10 0000000000000000 fffff80e
63a366f0 : OpenZFS+0x2fd432
ffffdf044b883b40 fffff80e
63a377c9 : 0000000000000001 00000000
00000000 0000001200000000 00000010
00000001 : nt!IofCallDriver+0x59
ffffdf044b883b80 fffff80e
63a36146 : ffffdf044b883c10 ffffdf04
4b883f20 0000000000000000 ffff8a09
7a2a1dc0 : FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x289
ffffdf044b883bf0 fffff800
0b82e209 : ffff8a09c0f1adc0 00000010
00000000 0000000000000000 ffffdf04
4b883f10 : FLTMGR!FltpDispatch+0xb6
ffffdf044b883c50 fffff80e
63a36173 : ffffcf01964ce180 ffffcf01
965dd180 0000000000000000 00000000
00000001 : nt!IofCallDriver+0x59
ffffdf044b883c90 fffff800
0b82e209 : ffff8a09c090e690 fffff800
0b82e247 0000000000000003 00000000
00000004 : FLTMGR!FltpDispatch+0xe3
ffffdf044b883cf0 fffff800
0b80d4a9 : ffff8a09c0f1adc0 ffff8a09
c090e690 ffff8a09c001d910 ffffdf04
4b883f20 : nt!IofCallDriver+0x59
ffffdf044b883d30 fffff800
0b923bce : 0000000000000000 ffffdf04
4b883dd0 ffff8a09c001d910 ffff8a09
c001d910 : nt!IoSynchronousPageWriteEx+0x139
ffffdf044b883d70 fffff800
0b867d10 : 0000000000000011 ffffbb85
14e0f5b0 0000000000000000 ffffa180
041eda60 : nt!MiIssueSynchronousFlush+0x72
ffffdf044b883df0 fffff800
0b89e2cb : ffffdf044b884130 ffffdf04
4b884268 0000000000000000 00000000
00000000 : nt!MiFlushSectionInternal+0x8d0
ffffdf044b8840c0 fffff800
0b89d328 : 0000000000000000 ffff8a09
be970080 0000000000001000 00000000
00000000 : nt!MmFlushSection+0x143
ffffdf044b884170 fffff800
0b92d9b6 : ffff8a0983eb2db8 00000000
00000000 ffff8a0900000000 00000000
00000000 : nt!CcFlushCachePriv+0x6e8
ffffdf044b8842d0 fffff800
15ce3c00 : 0000000000000000 00000000
00000000 fffff800162172ec ffff8a09
000011b9 : nt!CcFlushCache+0x16
ffffdf044b884310 fffff800
15fac761 : deadbeefdeadbeef ffff8a09
be970080 ffff8a0983eb2dd8 fffff800
15ce438e : OpenZFS+0x33c00
ffffdf044b8843b0 fffff800
15fafc5a : fffff80017560d80 fffff800
15ccb2c8 0000000000000000 00000000
00000000 : OpenZFS+0x2fc761
ffffdf044b884470 fffff800
15fad432 : 0000000000000000 00000000
000003a0 0000000000000030 ffff8a09
7a8a68a0 : OpenZFS+0x2ffc5a
ffffdf044b8845e0 fffff800
0b82e209 : ffffdf044b87f000 ffffdf04
4b8847a0 ffff8a09c090bad0 fffff80e
63a366f0 : OpenZFS+0x2fd432
ffffdf044b8846d0 fffff80e
63a377c9 : ffff8a09bd100001 ffffdf04
4b884780 ffff8a09be71ae30 ffff8a09
bd103db0 : nt!IofCallDriver+0x59
ffffdf044b884710 fffff80e
63a36146 : ffffdf044b8847a0 ffff8a09
c0f1adc0 ffffdf044b8847b0 ffff8a09
7a2a1dc0 : FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x289
ffffdf044b884780 fffff800
0b82e209 : ffff8a09c0f1adc0 fffff800
0b82e3e4 0000000000000001 fffff800
0b8536c4 : FLTMGR!FltpDispatch+0xb6
ffffdf044b8847e0 fffff80e
63a36173 : 0000000000000001 00000000
00000000 ffff8a09c0f1adc0 ffff8a09
bfe31080 : nt!IofCallDriver+0x59
ffffdf044b884820 fffff800
0b82e209 : ffff8a09c001d910 fffff800
0b8252bd 0000000000000000 00000000
00000000 : FLTMGR!FltpDispatch+0xe3
ffffdf044b884880 fffff800
0bdcadce : 0000000000000000 ffff8a09
c001d910 0000000000000000 ffff8a09
c090bad0 : nt!IofCallDriver+0x59
ffffdf044b8848c0 fffff800
0bdd5353 : ffff8a09bfe31080 00000000
00000000 ffffbb8500000000 ffff8a09
c001d8e0 : nt!IopCloseFile+0x15e
ffffdf044b884950 fffff800
0bddb63e : ffff8a09be970080 00000000
00000000 0000000000000000 00000000
00000000 : nt!ObCloseHandleTableEntry+0x543
ffffdf044b884a90 fffff800
0b9cd0f5 : ffff8a09be970080 00000000
00000000 ffffdf044b884b80 00000000
00000000 : nt!NtClose+0xde
ffffdf044b884b00 00007ffa
5b2809d4 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : nt!KiSystemServiceCopyEnd+0x25
000000b216589f38 00000000
00000000 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : 0x00007ffa`5b2809d4
SYMBOL_NAME: OpenZFS+30a54c
MODULE_NAME: OpenZFS IMAGE_NAME: OpenZFS.sys STACK_COMMAND: .cxr 0xffffdf044b882430 ; kb BUCKET_ID_FUNC_OFFSET: 30a54c FAILURE_BUCKET_ID: AV_OpenZFS!unknown_function OS_VERSION: 10.0.17763.1 BUILDLAB_STR: rs5_release
Perhaps useful:
STACK_TEXT:
ffffb083c3532fa0 fffff801
7e4d85dc : ffff96fcef000000 ffff96fc
ef5fffff 0000000000000001 ffffb207
e69ba770 : OpenZFS!sa_get_db+0x9 [C:\src\openzfs\module\zfs\sa.c @ 2154]
ffffb083c3532fb0 fffff801
7e2cb5da : 0000000000000000 ffffb207
e69ba770 ffffb207e40ccd00 00000000
00000000 : OpenZFS!zfs_write+0x13ac [C:\src\openzfs\module\zfs\zfs_vnops.c @ 681]
ffffb083c3533300 fffff801
7e2d0e0f : fffff8017f880d80 fffff801
7dfeb2c8 0000000000000000 00000000
00000008 : OpenZFS!fs_write+0x9aa [C:\src\openzfs\module\os\windows\zfs\zfs_vnops_windows.c @ 4136]
ffffb083c3533480 fffff801
7e2cd432 : 0000000000000000 00000000
000002c0 0000000000000030 ffffb207
a14548a0 : OpenZFS!fsDispatcher+0x175f [C:\src\openzfs\module\os\windows\zfs\zfs_vnops_windows.c @ 5727]
ffffb083c35335f0 fffff801
72a34209 : ffffb083c352e000 ffffb083
c35337b0 0000000000000000 fffff808
713866f0 : OpenZFS!dispatcher+0x292 [C:\src\openzfs\module\os\windows\zfs\zfs_vnops_windows.c @ 5857]
ffffb083c35336e0 fffff808
713877c9 : 0000000000000001 00000000
00000000 0000000000000000 fffff801
72a98bfd : nt!IofCallDriver+0x59
ffffb083c3533720 fffff808
71386146 : ffffb083c35337b0 ffffb207
e40ccdc0 0000000000000548 00000000
00000000 : FLTMGR!FltIsCallbackDataDirty+0x299
ffffb083c3533790 fffff801
72a34209 : ffffb207e40ccdc0 fffff801
72a596c4 ffffe9f4c092b248 ffffe9f4
fa7d3fff : FLTMGR!FltFreeCallbackData+0x556
ffffb083c35337f0 fffff808
71386173 : 0000000000000001 00000000
c8e65000 8000000000000867 fffff801
72d5104d : nt!IofCallDriver+0x59
ffffb083c3533830 fffff801
72a34209 : ffffb207e69ba610 ffffb207
e60f0000 0000000000000021 ffffb207
e403a080 : FLTMGR!FltFreeCallbackData+0x583
ffffb083c3533890 fffff801
72fe2841 : ffffb083c3533b80 ffffb207
e5d8b7a0 0000000000000001 ffffb207
e5d8b7a0 : nt!IofCallDriver+0x59
ffffb083c35338d0 fffff801
72fab84d : ffffb20700000000 00000000
00000000 ffffb083c3533b00 ffffb083
c3533b80 : nt!NtDeviceIoControlFile+0x221
ffffb083c3533980 fffff801
72bd30f5 : 0000000000000301 00000000
00000000 0000000000000000 00000000
00000000 : nt!NtWriteFile+0x8bd
ffffb083c3533a90 00007ff9
771608f4 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : nt!setjmpex+0x7f75
000000e7c5efb7f8 00000000
00000000 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : 0x00007ff9`771608f4
FAULTING_SOURCE_LINE: C:\src\openzfs\module\zfs\sa.c
FAULTING_SOURCE_LINE_NUMBER: 2154
STACK_COMMAND: .cxr 0xffffb083c35325b0 ; kb
EXCEPTION_CODE_STR: C71D7347
EXCEPTION_STR: WRONG_SYMBOLS
PROCESS_NAME: ntoskrnl.wrong.symbols.exe
System information
Type | Version/Name Windows Server | 2019 Distribution Name | Microsoft Distribution Version | Kernel Version | Architecture | OpenZFS Version | 2.1.6rc2
Describe the problem you're observing
After a few hours of tests with AJA System Test simulating video load to the top level of the pool, writes begin to fail, creating a folder in Explorer makes Explorer unresponsive, and copying files at the command line report that the file already exists. (See full error text below)
Describe how to reproduce the problem
Hours of write testing, possibly after the panic in the last issue.
Include any warning/errors/backtraces from the system logs
Mode LastWriteTime Length Name
-a---- 12/9/2022 11:59 PM 1710821113 FOO -a---- 12/9/2022 5:11 PM 1710821113 MEMORY.DMP -a---- 12/9/2022 5:11 PM 1710821113 COPYOFMEM
PS d:> cp .\FOO BAR cp : Cannot create "d:\BAR" because a file or directory with the same name already exists. At line:1 char:1
PS d:>