search

syncthing / syncthing

Open Source Continuous File Synchronization
https://syncthing.net/
Mozilla Public License 2.0
65.7k stars 4.32k forks source link

Windows x64 system crashes #9825

Open CmdrJorgs opened 6 days ago

CmdrJorgs commented 6 days ago

What happened?

Windows 11 PC has been crashing every other day. Bugcheck dumps show syncthing.exe (v1.28.0) is involved, see two of these dumps in the attached WinDbg outputs.

I'm seeing PAGE_FAULT_IN_NONPAGED_AREA and UNEXPECTED_KERNEL_MODE_TRAP (double fault) errors in the bugcheck dumps.

Syncthing version

v1.28.0

Platform & operating system

Windows 11 23H2 (build 22631.4460), Intel i9-9900K x64

Browser version

No response

Relevant log output

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

UNEXPECTED_KERNEL_MODE_TRAP (7f)
This means a trap occurred in kernel mode, and it's a trap of a kind
that the kernel isn't allowed to have/catch (bound trap) or that
is always instant death (double fault).  The first number in the
BugCheck params is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a *portion* of those codes:
If kv shows a taskGate
        use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
        use .trap on that value
Else
        .trap on the appropriate frame will show where the trap was taken
        (on x86, this will be the ebp that goes with the procedure KiTrap)
Endif
kb will then show the corrected stack.
Arguments:
Arg1: 0000000000000008, EXCEPTION_DOUBLE_FAULT
Arg2: ffff978039679e70
Arg3: fffff8060b1712d3
Arg4: fffff8060b174db3

Debugging Details:
------------------

KEY_VALUES_STRING: 1

    Key  : Analysis.CPU.mSec
    Value: 2375

    Key  : Analysis.Elapsed.mSec
    Value: 16862

    Key  : Analysis.IO.Other.Mb
    Value: 11

    Key  : Analysis.IO.Read.Mb
    Value: 1

    Key  : Analysis.IO.Write.Mb
    Value: 25

    Key  : Analysis.Init.CPU.mSec
    Value: 218

    Key  : Analysis.Init.Elapsed.mSec
    Value: 24336

    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 94

    Key  : Analysis.Version.DbgEng
    Value: 10.0.27725.1000

    Key  : Analysis.Version.Description
    Value: 10.2408.27.01 amd64fre

    Key  : Analysis.Version.Ext
    Value: 1.2408.27.1

    Key  : Bugcheck.Code.LegacyAPI
    Value: 0x1000007f

    Key  : Bugcheck.Code.TargetModel
    Value: 0x1000007f

    Key  : Dump.Attributes.AsUlong
    Value: 1808

    Key  : Dump.Attributes.DiagDataWrittenToHeader
    Value: 1

    Key  : Dump.Attributes.ErrorCode
    Value: 0

    Key  : Dump.Attributes.KernelGeneratedTriageDump
    Value: 1

    Key  : Dump.Attributes.LastLine
    Value: Dump completed successfully.

    Key  : Dump.Attributes.ProgressPercentage
    Value: 0

    Key  : Failure.Bucket
    Value: 0x7f_8_WppRecorder!memcpy

    Key  : Failure.Hash
    Value: {3a8093de-c142-f376-f47a-73b78c54bef7}

    Key  : Hypervisor.Enlightenments.ValueHex
    Value: 1417cf94

    Key  : Hypervisor.Flags.AnyHypervisorPresent
    Value: 1

    Key  : Hypervisor.Flags.ApicEnlightened
    Value: 1

    Key  : Hypervisor.Flags.ApicVirtualizationAvailable
    Value: 0

    Key  : Hypervisor.Flags.AsyncMemoryHint
    Value: 0

    Key  : Hypervisor.Flags.CoreSchedulerRequested
    Value: 0

    Key  : Hypervisor.Flags.CpuManager
    Value: 1

    Key  : Hypervisor.Flags.DeprecateAutoEoi
    Value: 0

    Key  : Hypervisor.Flags.DynamicCpuDisabled
    Value: 1

    Key  : Hypervisor.Flags.Epf
    Value: 0

    Key  : Hypervisor.Flags.ExtendedProcessorMasks
    Value: 1

    Key  : Hypervisor.Flags.HardwareMbecAvailable
    Value: 1

    Key  : Hypervisor.Flags.MaxBankNumber
    Value: 0

    Key  : Hypervisor.Flags.MemoryZeroingControl
    Value: 0

    Key  : Hypervisor.Flags.NoExtendedRangeFlush
    Value: 0

    Key  : Hypervisor.Flags.NoNonArchCoreSharing
    Value: 1

    Key  : Hypervisor.Flags.Phase0InitDone
    Value: 1

    Key  : Hypervisor.Flags.PowerSchedulerQos
    Value: 0

    Key  : Hypervisor.Flags.RootScheduler
    Value: 0

    Key  : Hypervisor.Flags.SynicAvailable
    Value: 1

    Key  : Hypervisor.Flags.UseQpcBias
    Value: 0

    Key  : Hypervisor.Flags.Value
    Value: 4853999

    Key  : Hypervisor.Flags.ValueHex
    Value: 4a10ef

    Key  : Hypervisor.Flags.VpAssistPage
    Value: 1

    Key  : Hypervisor.Flags.VsmAvailable
    Value: 1

    Key  : Hypervisor.RootFlags.AccessStats
    Value: 1

    Key  : Hypervisor.RootFlags.CrashdumpEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.CreateVirtualProcessor
    Value: 1

    Key  : Hypervisor.RootFlags.DisableHyperthreading
    Value: 0

    Key  : Hypervisor.RootFlags.HostTimelineSync
    Value: 1

    Key  : Hypervisor.RootFlags.HypervisorDebuggingEnabled
    Value: 0

    Key  : Hypervisor.RootFlags.IsHyperV
    Value: 1

    Key  : Hypervisor.RootFlags.LivedumpEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.MapDeviceInterrupt
    Value: 1

    Key  : Hypervisor.RootFlags.MceEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.Nested
    Value: 0

    Key  : Hypervisor.RootFlags.StartLogicalProcessor
    Value: 1

    Key  : Hypervisor.RootFlags.Value
    Value: 1015

    Key  : Hypervisor.RootFlags.ValueHex
    Value: 3f7

    Key  : Stack.Pointer
    Value: Invalid

BUGCHECK_CODE:  7f

BUGCHECK_P1: 8

BUGCHECK_P2: ffff978039679e70

BUGCHECK_P3: fffff8060b1712d3

BUGCHECK_P4: fffff8060b174db3

FILE_IN_CAB:  111424-15609-01.dmp

TAG_NOT_DEFINED_202b:  *** Unknown TAG in analysis list 202b

DUMP_FILE_ATTRIBUTES: 0x1808
  Kernel Generated Triage Dump

FAULTING_THREAD:  ffffd301a8aa0080

BAD_STACK_POINTER:  fffff8060b1712d3

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXNTFS: 1 (!blackboxntfs)

BLACKBOXPNP: 1 (!blackboxpnp)

BLACKBOXWINLOGON: 1

CUSTOMER_CRASH_COUNT:  1

PROCESS_NAME:  syncthing.exe

UNALIGNED_STACK_POINTER:  fffff8060b1712d3

SYMBOL_NAME:  WppRecorder!memcpy+73

MODULE_NAME: WppRecorder

IMAGE_NAME:  WppRecorder.sys

IMAGE_VERSION:  10.0.22621.1409

STACK_COMMAND:  .process /r /p 0xffffd301a8b33080; .thread 0xffffd301a8aa0080 ; kb

BUCKET_ID_FUNC_OFFSET:  73

FAILURE_BUCKET_ID:  0x7f_8_WppRecorder!memcpy

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

FAILURE_ID_HASH:  {3a8093de-c142-f376-f47a-73b78c54bef7}

Followup:     MachineOwner
---------

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

PAGE_FAULT_IN_NONPAGED_AREA (50)
Invalid system memory was referenced.  This cannot be protected by try-except.
Typically the address is just plain bad or it is pointing at freed memory.
Arguments:
Arg1: ffffc0890787d9b0, memory referenced.
Arg2: 0000000000000000, X64: bit 0 set if the fault was due to a not-present PTE.
    bit 1 is set if the fault was due to a write, clear if a read.
    bit 3 is set if the processor decided the fault was due to a corrupted PTE.
    bit 4 is set if the fault was due to attempted execute of a no-execute PTE.
    - ARM64: bit 1 is set if the fault was due to a write, clear if a read.
    bit 3 is set if the fault was due to attempted execute of a no-execute PTE.
Arg3: fffff800691d4d8f, If non-zero, the instruction address which referenced the bad memory
    address.
Arg4: 0000000000000002, (reserved)

Debugging Details:
------------------

KEY_VALUES_STRING: 1

    Key  : AV.Type
    Value: Read

    Key  : Analysis.CPU.mSec
    Value: 2093

    Key  : Analysis.Elapsed.mSec
    Value: 2392

    Key  : Analysis.IO.Other.Mb
    Value: 2

    Key  : Analysis.IO.Read.Mb
    Value: 1

    Key  : Analysis.IO.Write.Mb
    Value: 6

    Key  : Analysis.Init.CPU.mSec
    Value: 3250

    Key  : Analysis.Init.Elapsed.mSec
    Value: 25646

    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 103

    Key  : Analysis.Version.DbgEng
    Value: 10.0.27725.1000

    Key  : Analysis.Version.Description
    Value: 10.2408.27.01 amd64fre

    Key  : Analysis.Version.Ext
    Value: 1.2408.27.1

    Key  : Bugcheck.Code.LegacyAPI
    Value: 0x50

    Key  : Bugcheck.Code.TargetModel
    Value: 0x50

    Key  : Dump.Attributes.AsUlong
    Value: 1808

    Key  : Dump.Attributes.DiagDataWrittenToHeader
    Value: 1

    Key  : Dump.Attributes.ErrorCode
    Value: 0

    Key  : Dump.Attributes.KernelGeneratedTriageDump
    Value: 1

    Key  : Dump.Attributes.LastLine
    Value: Dump completed successfully.

    Key  : Dump.Attributes.ProgressPercentage
    Value: 0

    Key  : Failure.Bucket
    Value: IP_MISALIGNED_WppRecorder.sys

    Key  : Failure.Hash
    Value: {a9f7fe95-c676-0dcf-dbdf-709e0c33a748}

    Key  : Hypervisor.Enlightenments.ValueHex
    Value: 1417cf94

    Key  : Hypervisor.Flags.AnyHypervisorPresent
    Value: 1

    Key  : Hypervisor.Flags.ApicEnlightened
    Value: 1

    Key  : Hypervisor.Flags.ApicVirtualizationAvailable
    Value: 0

    Key  : Hypervisor.Flags.AsyncMemoryHint
    Value: 0

    Key  : Hypervisor.Flags.CoreSchedulerRequested
    Value: 0

    Key  : Hypervisor.Flags.CpuManager
    Value: 1

    Key  : Hypervisor.Flags.DeprecateAutoEoi
    Value: 0

    Key  : Hypervisor.Flags.DynamicCpuDisabled
    Value: 1

    Key  : Hypervisor.Flags.Epf
    Value: 0

    Key  : Hypervisor.Flags.ExtendedProcessorMasks
    Value: 1

    Key  : Hypervisor.Flags.HardwareMbecAvailable
    Value: 1

    Key  : Hypervisor.Flags.MaxBankNumber
    Value: 0

    Key  : Hypervisor.Flags.MemoryZeroingControl
    Value: 0

    Key  : Hypervisor.Flags.NoExtendedRangeFlush
    Value: 0

    Key  : Hypervisor.Flags.NoNonArchCoreSharing
    Value: 1

    Key  : Hypervisor.Flags.Phase0InitDone
    Value: 1

    Key  : Hypervisor.Flags.PowerSchedulerQos
    Value: 0

    Key  : Hypervisor.Flags.RootScheduler
    Value: 0

    Key  : Hypervisor.Flags.SynicAvailable
    Value: 1

    Key  : Hypervisor.Flags.UseQpcBias
    Value: 0

    Key  : Hypervisor.Flags.Value
    Value: 4853999

    Key  : Hypervisor.Flags.ValueHex
    Value: 4a10ef

    Key  : Hypervisor.Flags.VpAssistPage
    Value: 1

    Key  : Hypervisor.Flags.VsmAvailable
    Value: 1

    Key  : Hypervisor.RootFlags.AccessStats
    Value: 1

    Key  : Hypervisor.RootFlags.CrashdumpEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.CreateVirtualProcessor
    Value: 1

    Key  : Hypervisor.RootFlags.DisableHyperthreading
    Value: 0

    Key  : Hypervisor.RootFlags.HostTimelineSync
    Value: 1

    Key  : Hypervisor.RootFlags.HypervisorDebuggingEnabled
    Value: 0

    Key  : Hypervisor.RootFlags.IsHyperV
    Value: 1

    Key  : Hypervisor.RootFlags.LivedumpEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.MapDeviceInterrupt
    Value: 1

    Key  : Hypervisor.RootFlags.MceEnlightened
    Value: 1

    Key  : Hypervisor.RootFlags.Nested
    Value: 0

    Key  : Hypervisor.RootFlags.StartLogicalProcessor
    Value: 1

    Key  : Hypervisor.RootFlags.Value
    Value: 1015

    Key  : Hypervisor.RootFlags.ValueHex
    Value: 3f7

BUGCHECK_CODE:  50

BUGCHECK_P1: ffffc0890787d9b0

BUGCHECK_P2: 0

BUGCHECK_P3: fffff800691d4d8f

BUGCHECK_P4: 2

FILE_IN_CAB:  111124-16984-01.dmp

TAG_NOT_DEFINED_202b:  *** Unknown TAG in analysis list 202b

DUMP_FILE_ATTRIBUTES: 0x1808
  Kernel Generated Triage Dump

FAULTING_THREAD:  ffffac0fa89d4080

READ_ADDRESS: Unable to get NonPagedPoolStart
Unable to get NonPagedPoolEnd
Unable to get PagedPoolStart
Unable to get PagedPoolEnd
 ffffc0890787d9b0 

MM_INTERNAL_CODE:  2

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXNTFS: 1 (!blackboxntfs)

BLACKBOXPNP: 1 (!blackboxpnp)

BLACKBOXWINLOGON: 1

CUSTOMER_CRASH_COUNT:  1

PROCESS_NAME:  syncthing.exe

TRAP_FRAME:  ffffc08883c3ea70 -- (.trap 0xffffc08883c3ea70)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=000000000000e4ec rbx=0000000000000000 rcx=0000000083c3eca8
rdx=ffffc08883c3ed08 rsi=0000000000000000 rdi=0000000000000000
rip=fffff800691d4d8f rsp=ffffc08883c3ec08 rbp=fffff8006f5774e8
 r8=0000000000000004  r9=000000000000e518 r10=ffffac0f915fe000
r11=00000000000003fc r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz ac po nc
WppRecorder!memcpy+0x4f:
fffff800`691d4d8f ffc1            inc     ecx
Resetting default scope

MISALIGNED_IP: 
WppRecorder!memcpy+4f
fffff800`691d4d8f ffc1            inc     ecx

STACK_TEXT:  
ffffc088`83c3e848 fffff800`6ba7375c     : 00000000`00000050 ffffc089`0787d9b0 00000000`00000000 ffffc088`83c3ea70 : nt!KeBugCheckEx
ffffc088`83c3e850 fffff800`6b86c29c     : 00000000`000000b0 00000000`00000000 00000000`00000000 ffffc089`0787d9b0 : nt!MiSystemFault+0x23936c
ffffc088`83c3e950 fffff800`6ba25f7e     : ffffac0f`c11151a0 ffffac0f`bc0eef78 ffffac0f`950a89a0 00000000`00000010 : nt!MmAccessFault+0x29c
ffffc088`83c3ea70 fffff800`691d4d8f     : fffff800`691d12d3 ffffc088`83c3ed74 00000000`00000000 00000000`00000014 : nt!KiPageFault+0x37e
ffffc088`83c3ec08 fffff800`691d12d3     : ffffc088`83c3ed74 00000000`00000000 00000000`00000014 ffffac0f`915fe000 : WppRecorder!memcpy+0x4f
ffffc088`83c3ec10 fffff800`6f4af8e4     : 00000000`00000004 ffffc088`83c3ecf0 00000000`0e300e60 ffffc088`83c3ed18 : WppRecorder!WppAutoLogTrace+0x1a3
ffffc088`83c3ec70 fffff800`6f5ad6b9     : ffffc088`83c3ed74 ffffc088`83c3ed58 00000000`0000000a ffffc088`83c3ed58 : NDIS!WPP_RECORDER_SF_DqL+0x94
ffffc088`83c3ece0 fffff800`6f5ade56     : 00000000`0001028a 00000000`0000000c 00000000`00000004 00000000`00000288 : NDIS!ndisIfQueryLoopbackObject+0x59
ffffc088`83c3ed30 fffff800`6f4b06d6     : 00000000`00000008 ffffac0f`951488e0 ffffc088`83c3f110 ffffac0f`bc0eefa0 : NDIS!ndisNsiGetInterfaceRodEnumObject+0x286
ffffc088`83c3f030 fffff800`6f40472f     : 00000000`00000000 ffffac0f`8effff40 fffff800`6f575c98 00000000`00000000 : NDIS!ndisNsiEnumerateAllInterfaceInformation+0xa06
ffffc088`83c3f1a0 fffff800`858c1902     : 0000009d`8dfff2e0 00000000`00000000 0000009d`8dfff2e0 00000000`00000000 : NETIO!NsiEnumerateObjectsAllParametersEx+0x27f
ffffc088`83c3f3e0 fffff800`858c14c2     : 00000000`00000000 ffffac0f`a12fe3d0 ffffac0f`a12fe300 0000009d`8dfff2e0 : nsiproxy!NsippEnumerateObjectsAllParameters+0x282
ffffc088`83c3f5d0 fffff800`6b866e35     : ffffc088`83c3f6c1 00000000`00000000 00000000`00000000 ffffac0f`a8bbd5e0 : nsiproxy!NsippDispatch+0x82
ffffc088`83c3f620 fffff800`6bcbf140     : ffffac0f`a12fe300 00000000`00000002 ffffac0f`a8bbd5e0 ffffac0f`a8bbd5e0 : nt!IofCallDriver+0x55
ffffc088`83c3f660 fffff800`6bcbd9d0     : 00000000`00000000 00000000`00000070 ffffc088`83c3fa60 ffffac0f`a8bbd5e0 : nt!IopSynchronousServiceTail+0x1d0
ffffc088`83c3f710 fffff800`6bcbd2b6     : 00000000`00000001 ffffc088`83c3fa60 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x700
ffffc088`83c3f900 fffff800`6ba2a305     : ffffac0f`a89d4080 0000009d`8dfff1e8 ffffc088`83c3f988 00000000`00000000 : nt!NtDeviceIoControlFile+0x56
ffffc088`83c3f970 00007fff`97cf02b4     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25
0000009d`8dfff278 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007fff`97cf02b4

SYMBOL_NAME:  WppRecorder!memcpy+4f

MODULE_NAME: hardware

IMAGE_NAME:  hardware

IMAGE_VERSION:  10.0.22621.1409

STACK_COMMAND:  .process /r /p 0xffffac0fa8b74080; .thread 0xffffac0fa89d4080 ; kb

FAILURE_BUCKET_ID:  IP_MISALIGNED_WppRecorder.sys

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

FAILURE_ID_HASH:  {a9f7fe95-c676-0dcf-dbdf-709e0c33a748}

Followup:     MachineOwner
---------
rasa commented 6 days ago

@CmdrJorgs Would you mind installing an older version and see if the issue is still there? Try v1.27.12 which is the last version compiled using go 1.22. When you start the executable, either add --no-upgrade on the command line, or enter set STNOUPGRADE=1 before running the executable. This will keep syncthing from automatically upgrading itself.