pwsh/powershell VT quirk causes conhost crash when resizing while alt buffer is being used

[DHowett]

Repro:

while($true){ "`e[?1049h";sleep 0.25;"`e[?1049l"; sleep 0.25}

Resize the window horizontally while it's running.

Toggling the alt buffer from pwsh.exe or powershell.exe (specifically) causes us to UAF the outgoing SCREEN_INFORMATION here: https://github.com/microsoft/terminal/blob/71cbdc8a1f6b8da10d130c053489445e99a4fc13/src/host/_stream.cpp#L1038-L1040

[zadjii-msft]

note: We greatly reduced the number of places where this can trigger in #13352, but this quirk isn't entirely gone, so this bug persists.

[shuffle2]

fwiw I think I hit the same issue doing the following:

1. Run weechat in wsl2. Specifically, weechat 3.5 in ubuntu 22.04.1.
2. on the greeting screen (displays "WeeChat" in ascii art), resize Terminal until it crashes

It results in:

>   Microsoft.Terminal.Control.dll!Microsoft::Terminal::Core::Terminal::Write(std::basic_string_view<wchar_t,std::char_traits<wchar_t>> stringView) Line 486    C++
    Microsoft.Terminal.Control.dll!winrt::Microsoft::Terminal::Control::implementation::ControlCore::_connectionOutputHandler(const winrt::hstring & hstr) Line 1515    C++
    [Inline Frame] Microsoft.Terminal.Control.dll!winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler::{ctor}::__l1::<lambda_212_>::operator()(const winrt::hstring &) Line 507   C++
    Microsoft.Terminal.Control.dll!winrt::impl::delegate<winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler,`winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler::implementation<winrt::Microsoft::Terminal::Control::implementation::ControlCore,void (__cdecl winrt::Microsoft::Terminal::Control::implementation::ControlCore::*)(winrt::hstring const &)>'::`1'::<lambda_212_>>::Invoke(void * output) Line 174    C++
    [Inline Frame] TerminalConnection.dll!winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler::operator()(const winrt::param::hstring &) Line 451 C++
    TerminalConnection.dll!winrt::impl::invoke<winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler,std::wstring>(const winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler & delegate, const std::wstring & <args_0>) Line 5762 C++
    [Inline Frame] TerminalConnection.dll!winrt::event<winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler>::operator()(const std::wstring &) Line 5897   C++
    TerminalConnection.dll!winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::_OutputThread() Line 659   C++
    [Inline Frame] TerminalConnection.dll!winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::Start::__l3::<lambda_1>::operator()(void *) Line 362    C++
    TerminalConnection.dll!_Closure_wrapper_1bfe9aa2_14::<lambda_invoker_cdecl>(void * __p1) Line 368   C++
    kernel32.dll!BaseThreadInitThunk() Unknown
    ntdll.dll!RtlUserThreadStart() Unknown

  | Name | Value | Type -- | -- | -- | -- ▶ | this | 0x000002317a8988a0 {_pfnWriteInput={__this=0x000002317d41adc0 {...} } _pfnWarningBell=bind(0x00007fff6542dcd0 {Microsoft.Terminal.Control.dll!winrt::Microsoft::Terminal::Control::implementation::ControlCore::_terminalWarningBell(void)}, 0x000002317d41adc0 {...}) ...} | Microsoft::Terminal::Core::Terminal *   | cursorPosAfter | Variable is optimized away and not available. |   ▶ | cursorPosBefore | {x=0 X=0 y=32 ...} | til::point ▶ | lock | {_Pmtx=0x000002317a8989c0 {_next_ticket=435 _now_serving=434 } _Owns=true } | std::unique_lock ▶ | stringView | 0x000002316f86d060 L"\x1b[?1049l" | std::basic_string_view>

\x1b[?1049 appearing in the stringView seems suspicious.

This happens somewhat frequently while using WeeChat normally (i.e. not just looking at the greeting screen), so it's quite annoying, and has been occurring for a while.

I'm currently on Terminal 1.14.2282.0

I've just intsalled Microsoft.WindowsTerminal_Win11_1.15.2525.0_8wekyb3d8bbwe.msixbundle (hopefully that doesn't screw up the package on my system, which was previously just installed via Store...) and the crash is still present.

[DHowett]

Whoa, thanks for this! A repro that doesn't involve the powershell quirk (which we just removed!) makes this much more important, too.

[shuffle2]

I was going to take a crack at fixing this (at least locally), and I find that on my local build of current main branch (commit 1f19ed0cd2e04dd46852300c369960d6fa69a8fa), the crash does not occur. On the current release version (1.15.2875.0) I have installed, the crash still does occur.

What's the likelyhood this was actually fixed as opposed to being fixed by some difference in the build environment? What commit does 1.15.2875.0 relate to? There is no matching git tag.

[shuffle2]

Strangely I started getting a crash-on-resize again after running a round of Windows Update today (not sure if crash has same root cause though), and doing a clean rebuild of Terminal locally seems to have made the crash go away, again. I also updated VS2022 to current latest, so a lot of moving pieces, but... 🤨

[zadjii-msft]

Okay, coming back to this thread now that I'm back from leave and vacation.

I can't seem to repro this with weechat in main anymore. There've been some locking and lifetime changes over the last few months that might have fixed this. @shuffle2 if you can't repro this on main anymore, then I suggest we punt this out to the backlog. The set of folks with the busted PsReadline that we still need to quirk for who are resizing their terminal while entering the alt buffer is gonna be samll and only shrinking. We can elevate from the backlog if we see live hits here, but I don't think there are any.

Actually heck, @shuffle2's crash is a different thing anyways. The pwsh VT quirk only applies to conhost, and that stack is firmly in terminal code. I'm gonna punt and assume this was fixed. I vaguely recall some other resizing crash fixes, but 1.17 has been like, 4 months now.

[shuffle2]

@zadjii-msft I've just updated Windows and Terminal (from the store) again, and my crash (crash when resizing weechat output) still repros. The versions are: Terminal 1.18.2822.0 Windows 10.0.22621.2428

Although, getting the crash to occur isn't very deterministic. It seems to crash less frequently than before :/ Might need to try and compile with some debugging functionality and track it down eventually...unfortunately not much of a helpful update from me, here.

edit: actually, I'm probably hitting yet another crash, just happens to also be while doing my standard "resize weechat test". The current crash which has happened a few times looks like this:

!analyze output

``` KEY_VALUES_STRING: 1 Key : AV.Fault Value: Read Key : Analysis.CPU.mSec Value: 811 Key : Analysis.Elapsed.mSec Value: 7809 Key : Analysis.IO.Other.Mb Value: 42 Key : Analysis.IO.Read.Mb Value: 0 Key : Analysis.IO.Write.Mb Value: 149 Key : Analysis.Init.CPU.mSec Value: 171 Key : Analysis.Init.Elapsed.mSec Value: 9188 Key : Analysis.Memory.CommitPeak.Mb Value: 168 Key : Failure.Bucket Value: INVALID_POINTER_READ_c0000005_Microsoft.Terminal.Control.dll!Microsoft::Terminal::Core::Terminal::Write Key : Failure.Hash Value: {39be1696-4bbf-0321-1166-d994398e7f26} Key : Timeline.OS.Boot.DeltaSec Value: 508 Key : Timeline.Process.Start.DeltaSec Value: 21 Key : WER.OS.Branch Value: ni_release Key : WER.OS.Version Value: 10.0.22621.1 Key : WER.Process.Version Value: 1.18.2310.9002 FILE_IN_CAB: WindowsTerminal.exe.19076.dmp NTGLOBALFLAG: 400 PROCESS_BAM_CURRENT_THROTTLED: 0 PROCESS_BAM_PREVIOUS_THROTTLED: 0 APPLICATION_VERIFIER_FLAGS: 0 CONTEXT: (.ecxr) rax=000000ace7f0f8c0 rbx=000000250000000a rcx=1bb06e0f88a70000 rdx=000002216288c534 rsi=00000221628ad940 rdi=000002216139d1f0 rip=00007ffd69f0a495 rsp=000000ace7f0f9c0 rbp=000002216139d308 r8=000000000000000a r9=0000000000000000 r10=000000ace7f0f8c0 r11=0010000004040001 r12=00000221613a3898 r13=0000000000000000 r14=000000ace7f0fa50 r15=0000000000000000 iopl=0 nv up ei pl nz na pe nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 Microsoft_Terminal_Control!til::point::operator!= [inlined in Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x7d]: 00007ffd`69f0a495 483b9e10010000 cmp rbx,qword ptr [rsi+110h] ds:00000221`628ada50=???????????????? Resetting default scope EXCEPTION_RECORD: (.exr -1) ExceptionAddress: 00007ffd69f0a495 (Microsoft_Terminal_Control!til::point::operator!=) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 0000000000000000 Parameter[1]: 00000221628ada50 Attempt to read from address 00000221628ada50 PROCESS_NAME: WindowsTerminal.exe READ_ADDRESS: 00000221628ada50 ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. EXCEPTION_CODE_STR: c0000005 EXCEPTION_PARAMETER1: 0000000000000000 EXCEPTION_PARAMETER2: 00000221628ada50 STACK_TEXT: 000000ac`e7f0f9c0 00007ffd`69f0a397 : 00000000`00000024 00007ffd`6c3780ea 00000221`5f032240 00000000`00000000 : Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x7d 000000ac`e7f0fa30 00007ffd`69f1d8e2 : 00000221`613a3898 00000000`00000000 00000221`6135b6d8 00000221`613a3878 : Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::_connectionOutputHandler+0x37 000000ac`e7f0fa80 00007ffd`6c378b1d : 00000221`6288c520 00007ffd`00000012 00000859`00000001 00000221`613a3870 : Microsoft_Terminal_Control!winrt::impl::delegate'::`1':: >::Invoke+0x32 000000ac`e7f0fad0 00007ffd`6c3786d4 : 00000000`00000000 00004000`00000000 00000000`00000000 000000ac`e7f0fbb0 : TerminalConnection!winrt::impl::invoke,std::allocator > >+0x7d 000000ac`e7f0fb40 00007ffd`6c3792de : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : TerminalConnection!winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::_OutputThread+0x154 000000ac`e7f0fc00 00007ffd`9764257d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : TerminalConnection!`winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::Start'::`3'::::+0xe 000000ac`e7f0fc30 00007ffd`98c4aa78 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x1d 000000ac`e7f0fc60 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x28 FAULTING_SOURCE_LINE: C:\__w\1\s\src\cascadia\TerminalCore\Terminal.cpp FAULTING_SOURCE_FILE: C:\__w\1\s\src\cascadia\TerminalCore\Terminal.cpp FAULTING_SOURCE_LINE_NUMBER: 493 FAULTING_SOURCE_CODE: 52: } 53: 54: constexpr bool operator!=(const point rhs) const noexcept 55: { > 56: return __builtin_memcmp(this, &rhs, sizeof(rhs)) != 0; 57: } 58: 59: constexpr explicit operator bool() const noexcept 60: { 61: return (x > 0) & (y > 0); SYMBOL_NAME: Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+7d MODULE_NAME: Microsoft_Terminal_Control IMAGE_NAME: Microsoft.Terminal.Control.dll STACK_COMMAND: ~88s; .ecxr ; kb FAILURE_BUCKET_ID: INVALID_POINTER_READ_c0000005_Microsoft.Terminal.Control.dll!Microsoft::Terminal::Core::Terminal::Write OS_VERSION: 10.0.22621.1 BUILDLAB_STR: ni_release OSPLATFORM_TYPE: x64 OSNAME: Windows 10 IMAGE_VERSION: 1.18.2310.9002 FAILURE_ID_HASH: {39be1696-4bbf-0321-1166-d994398e7f26} ```

faulting context

``` 0:088> k # Child-SP RetAddr Call Site 00 000000ac`e7f0e4e8 00007ffd`9658f969 ntdll!NtWaitForMultipleObjects+0x14 01 000000ac`e7f0e4f0 00007ffd`9658f86e KERNELBASE!WaitForMultipleObjectsEx+0xe9 02 000000ac`e7f0e7d0 00007ffd`976a27f7 KERNELBASE!WaitForMultipleObjects+0xe 03 000000ac`e7f0e810 00007ffd`976a2236 kernel32!WerpReportFaultInternal+0x587 04 000000ac`e7f0e930 00007ffd`9668c92b kernel32!WerpReportFault+0xbe 05 000000ac`e7f0e970 00007ffd`98c98b3d KERNELBASE!UnhandledExceptionFilter+0x3db 06 000000ac`e7f0ea90 00007ffd`98c7f1b7 ntdll!RtlUserThreadStart$filt$0+0xac 07 000000ac`e7f0ead0 00007ffd`98c9449f ntdll!_C_specific_handler+0x97 08 000000ac`e7f0eb40 00007ffd`98c0e456 ntdll!RtlpExecuteHandlerForException+0xf 09 000000ac`e7f0eb70 00007ffd`98c9348e ntdll!RtlDispatchException+0x286 0a 000000ac`e7f0f2c0 00007ffd`69f0a495 ntdll!KiUserExceptionDispatch+0x2e 0b 000000ac`e7f0f9c0 00007ffd`69f0a397 Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x7d [C:\__w\1\s\src\cascadia\TerminalCore\Terminal.cpp @ 493] 0c 000000ac`e7f0fa30 00007ffd`69f1d8e2 Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::_connectionOutputHandler+0x37 [C:\__w\1\s\src\cascadia\TerminalControl\ControlCore.cpp @ 1804] 0d (Inline Function) --------`-------- Microsoft_Terminal_Control!winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler::{ctor}::__l1::::operator()+0x17 [C:\__w\1\s\src\cascadia\TerminalControl\Generated Files\winrt\Microsoft.Terminal.TerminalConnection.h @ 534] 0e 000000ac`e7f0fa80 00007ffd`6c378b1d Microsoft_Terminal_Control!winrt::impl::delegate'::`1':: >::Invoke+0x32 [C:\__w\1\s\src\cascadia\TerminalControl\Generated Files\winrt\Microsoft.Terminal.TerminalConnection.h @ 186] 0f (Inline Function) --------`-------- TerminalConnection!winrt::Microsoft::Terminal::TerminalConnection::TerminalOutputHandler::operator()+0x3d [C:\__w\1\s\src\cascadia\TerminalConnection\Generated Files\winrt\Microsoft.Terminal.TerminalConnection.h @ 478] 10 000000ac`e7f0fad0 00007ffd`6c3786d4 TerminalConnection!winrt::impl::invoke,std::allocator > >+0x7d [C:\__w\1\s\src\cascadia\TerminalConnection\Generated Files\winrt\base.h @ 5890] 11 (Inline Function) --------`-------- TerminalConnection!winrt::event::operator()+0x54 [C:\__w\1\s\src\cascadia\TerminalConnection\Generated Files\winrt\base.h @ 6043] 12 000000ac`e7f0fb40 00007ffd`6c3792de TerminalConnection!winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::_OutputThread+0x154 [C:\__w\1\s\src\cascadia\TerminalConnection\ConptyConnection.cpp @ 711] 13 (Inline Function) --------`-------- TerminalConnection!winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::Start::__l3::::operator()+0xa [C:\__w\1\s\src\cascadia\TerminalConnection\ConptyConnection.cpp @ 427] 14 000000ac`e7f0fc00 00007ffd`9764257d TerminalConnection!`winrt::Microsoft::Terminal::TerminalConnection::implementation::ConptyConnection::Start'::`3'::::+0xe [C:\__w\1\s\src\cascadia\TerminalConnection\ConptyConnection.cpp @ 430] 15 000000ac`e7f0fc30 00007ffd`98c4aa78 kernel32!BaseThreadInitThunk+0x1d 16 000000ac`e7f0fc60 00000000`00000000 ntdll!RtlUserThreadStart+0x28 0:088> .frame /c b 0b 000000ac`e7f0f9c0 00007ffd`69f0a397 Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x7d [C:\__w\1\s\src\cascadia\TerminalCore\Terminal.cpp @ 493] rax=0000000000000000 rbx=000000250000000a rcx=0000000000000001 rdx=0000000100000000 rsi=00000221628ad940 rdi=000002216139d1f0 rip=00007ffd69f0a495 rsp=000000ace7f0f9c0 rbp=000002216139d308 r8=000002215f17a280 r9=0000000000000001 r10=0000000000000838 r11=00001b1c89ff7e37 r12=00000221613a3898 r13=0000000000000000 r14=000000ace7f0fa50 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 Microsoft_Terminal_Control!til::point::operator!= [inlined in Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x7d]: 00007ffd`69f0a495 483b9e10010000 cmp rbx,qword ptr [rsi+110h] ds:00000221`628ada50=???????????????? ```

faulting code

``` Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write: 00007ffd`69f0a418 48895c2418 mov qword ptr [rsp+18h], rbx 00007ffd`69f0a41d 48896c2420 mov qword ptr [rsp+20h], rbp 00007ffd`69f0a422 56 push rsi 00007ffd`69f0a423 57 push rdi 00007ffd`69f0a424 4156 push r14 00007ffd`69f0a426 4883ec50 sub rsp, 50h 00007ffd`69f0a42a 488b05dfdb1a00 mov rax, qword ptr [Microsoft_Terminal_Control!__security_cookie (7ffd6a0b8010)] 00007ffd`69f0a431 4833c4 xor rax, rsp 00007ffd`69f0a434 4889442440 mov qword ptr [rsp+40h], rax 00007ffd`69f0a439 4c8bf2 mov r14, stringView (rdx) 00007ffd`69f0a43c 488bf9 mov rdi, this (rcx) 00007ffd`69f0a43f 0f57c0 xorps xmm0, xmm0 00007ffd`69f0a442 0f11442430 movups xmmword ptr [lock{._Pmtx(!!)} (rsp+30h)], xmm0 00007ffd`69f0a447 488da918010000 lea rbp, [rcx+118h] 00007ffd`69f0a44e 48896c2430 mov qword ptr [lock{._Pmtx} (rsp+30h)], lock (rbp) 00007ffd`69f0a453 488bcd mov rcx, rbp 00007ffd`69f0a456 e879000000 call Microsoft_Terminal_Control!til::recursive_ticket_lock::lock (7ffd69f0a4d4) 00007ffd`69f0a45b c644243801 mov byte ptr [lock._Owns (rsp+38h)], 1 00007ffd`69f0a460 488bb720080000 mov rsi, qword ptr [this->_altBuffer{._Mypair._Myval2} (rdi+820h)] 00007ffd`69f0a467 4885f6 test rsi, rsi 00007ffd`69f0a46a 7507 jne Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x5b (7ffd69f0a473) 00007ffd`69f0a46c 488bb718080000 mov rsi, qword ptr [this->_mainBuffer{._Mypair._Myval2} (rdi+818h)] 00007ffd`69f0a473 488b9e10010000 mov rbx, qword ptr [rsi+110h] 00007ffd`69f0a47a 410f2806 movaps xmm0, xmmword ptr [stringView{._Mydata(!!)} (r14)] 00007ffd`69f0a47e 660f7f442420 movdqa xmmword ptr [rsp+20h], xmm0 00007ffd`69f0a484 488d542420 lea rdx, [rsp+20h] 00007ffd`69f0a489 488b8fc8060000 mov rcx, qword ptr [this->_stateMachine{._Mypair._Myval2} (rdi+6C8h)] 00007ffd`69f0a490 e8db000000 call Microsoft_Terminal_Control!Microsoft::Console::VirtualTerminal::StateMachine::ProcessString (7ffd69f0a570) 00007ffd`69f0a495 483b9e10010000 cmp cursorPosBefore (rbx), qword ptr [rsi+110h] <--------------- rsi is unmapped (probably UAF) 00007ffd`69f0a49c 752a jne Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0xb0 (7ffd69f0a4c8) 00007ffd`69f0a49e 488bcd mov rcx, rbp 00007ffd`69f0a4a1 e84a140000 call Microsoft_Terminal_Control!til::recursive_ticket_lock::unlock (7ffd69f0b8f0) 00007ffd`69f0a4a6 488b4c2440 mov rcx, qword ptr [rsp+40h] 00007ffd`69f0a4ab 4833cc xor rcx, rsp 00007ffd`69f0a4ae e83d1c0300 call Microsoft_Terminal_Control!__security_check_cookie (7ffd69f3c0f0) 00007ffd`69f0a4b3 4c8d5c2450 lea r11, [rsp+50h] 00007ffd`69f0a4b8 498b5b30 mov cursorPosBefore (rbx), qword ptr [r11+30h] 00007ffd`69f0a4bc 498b6b38 mov rbp, qword ptr [r11+38h] 00007ffd`69f0a4c0 498be3 mov rsp, r11 00007ffd`69f0a4c3 415e pop stringView (r14) 00007ffd`69f0a4c5 5f pop this (rdi) 00007ffd`69f0a4c6 5e pop rsi 00007ffd`69f0a4c7 c3 ret 00007ffd`69f0a4c8 488bcf mov rcx, this (rdi) 00007ffd`69f0a4cb e844140000 call Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::_NotifyTerminalCursorPositionChanged (7ffd69f0b914) 00007ffd`69f0a4d0 ebcc jmp Microsoft_Terminal_Control!Microsoft::Terminal::Core::Terminal::Write+0x86 (7ffd69f0a49e) 00007ffd`69f0a4d2 cc int 3 00007ffd`69f0a4d3 cc int 3 ```