Bugcheck in ovpn_dco!OvpnTxBufferPoolGet

lstipakov commented 10 months ago

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 0000000000000020, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, bitfield :
    bit 0 : value 0 = read operation, 1 = write operation
    bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
Arg4: fffff8030f6c15a4, address which referenced memory

Debugging Details:
------------------

KEY_VALUES_STRING: 1

    Key  : Analysis.CPU.mSec
    Value: 2436

    Key  : Analysis.DebugAnalysisManager
    Value: Create

    Key  : Analysis.Elapsed.mSec
    Value: 9260

    Key  : Analysis.Init.CPU.mSec
    Value: 17421

    Key  : Analysis.Init.Elapsed.mSec
    Value: 6418256

    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 115

    Key  : WER.OS.Branch
    Value: ni_release_svc_prod3

    Key  : WER.OS.Timestamp
    Value: 2023-06-22T09:51:00Z

    Key  : WER.OS.Version
    Value: 10.0.22621.1928

FILE_IN_CAB:  090123-14234-01.dmp

TAG_NOT_DEFINED_202b:  *** Unknown TAG in analysis list 202b

BUGCHECK_CODE:  a

BUGCHECK_P1: 20

BUGCHECK_P2: 2

BUGCHECK_P3: 0

BUGCHECK_P4: fffff8030f6c15a4

READ_ADDRESS: Unable to get NonPagedPoolStart
Unable to get NonPagedPoolEnd
Unable to get PagedPoolStart
Unable to get PagedPoolEnd
 0000000000000020 

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXNTFS: 1 (!blackboxntfs)

BLACKBOXPNP: 1 (!blackboxpnp)

BLACKBOXWINLOGON: 1

CUSTOMER_CRASH_COUNT:  1

PROCESS_NAME:  System

TRAP_FRAME:  fffffe893a787260 -- (.trap 0xfffffe893a787260)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000
rdx=00000000000002a0 rsi=0000000000000000 rdi=0000000000000000
rip=fffff8030f6c15a4 rsp=fffffe893a7873f0 rbp=fffffe893a787551
 r8=00000000000000d0  r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei ng nz na po nc
nt!MmBuildMdlForNonPagedPool+0x14:
fffff803`0f6c15a4 488b7920        mov     rdi,qword ptr [rcx+20h] ds:00000000`00000020=????????????????
Resetting default scope

STACK_TEXT:  
fffffe89`3a787118 fffff803`0f8468a9     : 00000000`0000000a 00000000`00000020 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
fffffe89`3a787120 fffff803`0f841f34     : 00000000`00000000 fffff803`000000d0 00000000`00000002 00000000`00000000 : nt!KiBugCheckDispatch+0x69
fffffe89`3a787260 fffff803`0f6c15a4     : 00000000`00000008 00000000`000002b3 00000000`00000656 00000000`000000b8 : nt!KiPageFault+0x474
fffffe89`3a7873f0 fffff803`2a252e69     : 00000000`000002b3 fffffe89`3a787510 fffffe89`3a787510 fffffe89`3a787551 : nt!MmBuildMdlForNonPagedPool+0x14
fffffe89`3a787480 fffff803`2a257bf7     : ffff9285`41a262f0 fffff803`2a27294e fffffe89`3a7875bc 00000000`00000000 : ovpn_dco!OvpnTxBufferPoolGet+0x4d [c:\Users\lev\Projects\ovpn-dco-win\bufferpool.cpp @ 167] 
fffffe89`3a7874c0 fffff803`2a257a17     : ffff9285`41c6b398 ffff9285`41bd0a90 fffffe89`3a787610 fffffe89`3a7875f0 : ovpn_dco!OvpnTxProcessPacket+0x63 [c:\Users\lev\Projects\ovpn-dco-win\txqueue.cpp @ 49] 
fffffe89`3a7875b0 fffff803`2a28d8aa     : ffff9285`4462d0d0 fffffe89`3a787900 ffff9285`41bd0a90 fffff803`2a273995 : ovpn_dco!OvpnEvtTxQueueAdvance+0xe7 [c:\Users\lev\Projects\ovpn-dco-win\txqueue.cpp @ 154] 
fffffe89`3a787670 fffff803`2a29d915     : ffff9285`41c6b1a0 00000000`000005c0 ffff9285`41c6b1a0 ffff9285`41c6b640 : NetAdapterCx!NetClientQueueAdvance+0x2a [minio\netcx\adapter\nxqueue.cpp @ 50] 
fffffe89`3a7876a0 fffff803`2a29d4c6     : ffff9285`41c6b2c0 ffff9285`41c6b5c0 fffffe89`3a7879d0 ffff9285`41c6b5a0 : NetAdapterCx!NxTxXlat::YieldToNetAdapter+0xa5 [minio\netcx\translator\nxtxxlat.cpp @ 478] 
fffffe89`3a7876d0 fffff803`2a29d209     : 00000000`00000000 fffffe89`3a7879f0 ffff9285`41c6b1e8 00000000`00000000 : NetAdapterCx!NxTxXlat::TransmitNbls+0x196 [minio\netcx\translator\nxtxxlat.cpp @ 219] 
fffffe89`3a7878c0 fffff803`2a2a785a     : ffff9285`41c6b1e8 ffff9285`41c6c4d8 00000000`00000004 00000000`00000000 : NetAdapterCx!EvtTxPollQueueStarted+0x9 [minio\netcx\translator\nxtxxlat.cpp @ 185] 
fffffe89`3a7878f0 fffff803`2a2a7db1     : ffff9285`250fe040 fffff803`0f860c76 00000000`00000000 00000000`00000080 : NetAdapterCx!ExecutionContext::Poll+0x19e [minio\netcx\ec\lib\executioncontext.cpp @ 890] 
fffffe89`3a787a40 fffff803`2a2a7c47     : 00000000`00000003 00000000`00000080 ffff9285`2a1b7100 fffff780`00000014 : NetAdapterCx!ExecutionContext::RunLockHeld+0x13d [minio\netcx\ec\lib\executioncontext.cpp @ 1093] 
fffffe89`3a787a80 fffff803`2a2a7561     : ffff9285`41c6c010 ffff9285`41c6c010 ffff9285`254c9000 ffff9285`2a1b7100 : NetAdapterCx!ExecutionContext::RunLockNotHeld+0x3b [minio\netcx\ec\lib\executioncontext.cpp @ 1010] 
fffffe89`3a787ac0 fffff803`2a2c7639     : ffff9285`4497b040 fffff803`2a2c7630 00000000`00000000 00000000`0443315f : NetAdapterCx!ExecutionContext::WorkerThreadRoutine+0x65 [minio\netcx\ec\lib\executioncontext.cpp @ 762] 
fffffe89`3a787b00 fffff803`0f724e47     : ffff9285`4497b040 fffff803`2a2c7630 ffff9285`41c6c010 006fe47f`b19bbdff : NetAdapterCx!ExecutionContext::Thread+0x9 [minio\netcx\ec\lib\executioncontext.cpp @ 737] 
fffffe89`3a787b30 fffff803`0f8361b4     : ffffbd80`77051180 ffff9285`4497b040 fffff803`0f724df0 00000000`00000000 : nt!PspSystemThreadStartup+0x57
fffffe89`3a787b80 00000000`00000000     : fffffe89`3a788000 fffffe89`3a781000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x34

FAULTING_SOURCE_LINE:  c:\Users\lev\Projects\ovpn-dco-win\bufferpool.cpp

FAULTING_SOURCE_FILE:  c:\Users\lev\Projects\ovpn-dco-win\bufferpool.cpp

FAULTING_SOURCE_LINE_NUMBER:  167

FAULTING_SOURCE_CODE:  
   163: 
   164:     (*buffer)->Mdl = IoAllocateMdl(*buffer, ((OVPN_BUFFER_POOL_IMPL*)handle)->ItemSize, FALSE, FALSE, NULL);
   165:     MmBuildMdlForNonPagedPool((*buffer)->Mdl);
   166: 
>  167:     (*buffer)->Pool = handle;
   168: 
   169:     (*buffer)->Data = (*buffer)->Head + OVPN_BUFFER_HEADROOM;
   170:     (*buffer)->Tail = (*buffer)->Data;
   171: 
   172:     (*buffer)->Len = 0;

SYMBOL_NAME:  ovpn_dco!OvpnTxBufferPoolGet+4d

MODULE_NAME: ovpn_dco

IMAGE_NAME:  ovpn-dco.sys

STACK_COMMAND:  .cxr; .ecxr ; kb

BUCKET_ID_FUNC_OFFSET:  4d

FAILURE_BUCKET_ID:  AV_ovpn_dco!OvpnTxBufferPoolGet

OS_VERSION:  10.0.22621.1928

BUILDLAB_STR:  ni_release_svc_prod3

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

FAILURE_ID_HASH:  {77c9c180-2793-eebf-3c3b-97fd67f64bbe}

Followup:     MachineOwner
---------

lstipakov commented 10 months ago

From what I see, TX pool is unreasonably large:

3: kd> dx ((ovpn_dco!OVPN_BUFFER_POOL_IMPL *)0xffff928541996b90)
((ovpn_dco!OVPN_BUFFER_POOL_IMPL *)0xffff928541996b90)                 : 0xffff928541996b90 [Type: OVPN_BUFFER_POOL_IMPL *]
    [+0x000] ListHead         [Type: _LIST_ENTRY]
    [+0x010] Lock             : 0x0 [Type: unsigned __int64]
    [+0x018] ItemSize         : 0x656 [Type: unsigned int]
    [+0x01c] PoolSize         : 1832047 [Type: long]
    [+0x020] Context          : 0xffff928541a262f0 [Type: void *]
    [+0x028] Tag              : 0xfffff8032a25b280 : "tx" [Type: char *]

lstipakov commented 9 months ago

Portal incident id: 46108914

OpenVPN / ovpn-dco-win

Bugcheck in ovpn_dco!OvpnTxBufferPoolGet #47