PowerShell crash caused by WindowsTerminal.exe exception

Tina-TianluZhao commented 1 month ago

Windows Terminal version

1.20.11781.0

Windows build number

10.0.22631.3880

Other Software

No response

Steps to reproduce

This is an unstable but high frequency issue repro on Windows machine in pipeline. It is serious affected us. Hope you guys can fix it or give us some suggestions, it will be great appreciated. Thanks a lot.

This issue is often triggered by following commands run in PowerShell:

C:\Program Files (x86)\Android\android-sdk\cmdline-tools\11.0\bin\sdkmanager.bat' --verbose --install 'system-images;android-34;google_apis_playstore;x86_64'
C:\Program Files\Microsoft Visual Studio\2022\Main\Common7\IDE\devenv.exe /resetsettings "General.vssettings" /command "File.Exit"

The exceptions information details are as below, I have attached the dump file of "exception 2", and I will continue tracking and will attach once get the dump of "exception 1". Exception 1: _Faultinq application name: WindowsTerminal.exe, version: 1.20.2406.26001, time stamp: 0x667c7d41 Faultinq module name: ucrtbase.dll, version: 10.0.22621.3593, time stamp: 0x10c46e71 Exception code: 0xc0000409 Fault offset: 0x000000000007f6fe Faultinq process id: 0x0xx34EC Faultinq application start time: 0x0x1DADD30DC690A41 Faultinq application path: D:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.20.11781.0_x648wekyb3d8bbwe\WindowsTerminal.exe

Exception 2: _Faultinq application name: WindowsTerminal.exe, version: 1.20.2406.26001, time stamp: 0x667c7d41 Faulting module name: TerminalApp.dil, version: 1.20.2406.26001, time stamp: 0x667c7cd3 Exception code: 0xc0000005 Fault offset: 0x000000000003a29c Faultinq process id: (c0x1480 Faultinq application start time: 0x0x1DADE40231FE675 Faultinq application path: D:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.20.11781.0_x64_8wekyb3d8bbwe\WindowsTerminal.exe Faultinq module path: D:\Proqram Files\WindowsApps\Microsoft.WindowsTerminal_1.20.11781.0_x64_8wekyb3d8bbwe\TerminalApp.dll Report Id: e30d18c0-8cd0-4d42-a168-6b7e4b775512 Faulting package full name: Microsoft.WindowsTerminal_1.20.11781.0_x648wekyb3d8bbwe Faultinq package-relative application ID: App

Expected Behavior

No exception

Actual Behavior

When running commands, other PowerShell windows that were originally opened will be closed. Screenshot 2024-07-24 101055

lhecker commented 1 month ago

Your user contains "VSM", so I guess it's a VM? Perhaps with a virtualized GPU? In that case it could be #17536.

You can work around the crash for now by changing the default terminal application in the VM. Open Windows Terminal and press Ctrl+, (Ctrl+Comma) to open the settings. Then choose the Windows Console Host and click Save:

Alternatively, here's how to do this by changing registry keys: https://support.microsoft.com/en-us/windows/command-prompt-and-windows-powershell-for-windows-11-6453ce98-da91-476f-8651-5c14d5777c20#ID0EFD

Tina-TianluZhao commented 1 month ago

Thanks @lhecker , I will set up multiple machines with your workaround and observe if it resolves the issue. I'd like to confirm if this workaround only works for VM, since our machines are on-prem.

lhecker commented 1 month ago

This works for any Windows machine.

If this issue also occurs on non-virtualized Windows machines (on regular PCs), please let us know.

Tina-TianluZhao commented 1 month ago

@lhecker , the machines we used are regular PC, they are non-virtualized. As for the confusing name prefix 'VSM', I guess it may stands for "Visual Studio Maui?/Machine?" :relaxed:

lhecker commented 1 month ago

Can you please check %LOCALAPPDATA%\CrashDumps and see if there are files named WindowsTerminal in there? If there are any, and if you don't mind sharing the dumps with us, could you please upload one (or a few) of them somewhere (OneDrive, Dropbox, etc.) and then share a link to them with us? You can share a link privately with us by sending a mail to <my github username>@microsoft.com.

Tina-TianluZhao commented 1 month ago

Sure thing, once I get the dump I will share it with you via email. Actually, the dump has already been generated and I don't have permissions to operate it, so I have to wait for the other team to transfer it to me. Please wait.

Tina-TianluZhao commented 1 month ago

Hi @lhecker , the dump file has been sent to you via email.

lhecker commented 3 weeks ago

I've had a chance to look at the crash dump now. It was very helpful!

Here's the stack trace:

0:000> kc
 # Call Site
00 TerminalApp!winrt::impl::consume_Windows_UI_Core_ICoreDispatcher<winrt::Windows::UI::Core::ICoreDispatcher>::RunAsync
01 TerminalApp!wil::details::dispatcher_RunAsync::Schedule
02 TerminalApp!`wil::resume_foreground<winrt::Windows::UI::Core::CoreDispatcher>'::`2'::awaitable::await_suspend
03 TerminalApp!winrt::TerminalApp::implementation::TerminalWindow::UpdateSettings$_ResumeCoro$1
04 TerminalApp!winrt::TerminalApp::implementation::TerminalWindow::UpdateSettings$_InitCoro$2
05 TerminalApp!winrt::TerminalApp::implementation::TerminalWindow::UpdateSettings
06 TerminalApp!winrt::TerminalApp::implementation::TerminalWindow::UpdateSettingsHandler
07 TerminalApp!winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs>::{ctor}::__l1::<lambda_475_>::operator()
08 TerminalApp!winrt::impl::delegate<winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs>,`winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs>::implementation<winrt::TerminalApp::implementation::TerminalWindow,void (__cdecl winrt::TerminalApp::implementation::TerminalWindow::*)(winrt::Windows::Foundation::IInspectable const &,winrt::TerminalApp::SettingsLoadEventArgs const &)>'::`1'::<lambda_475_> >::Invoke
09 TerminalApp!winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs>::operator()
0a TerminalApp!winrt::impl::invoke<winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs>,winrt::TerminalApp::implementation::AppLogic,winrt::TerminalApp::implementation::SettingsLoadEventArgs>
0b TerminalApp!winrt::event<winrt::Windows::Foundation::TypedEventHandler<winrt::Windows::Foundation::IInspectable,winrt::TerminalApp::SettingsLoadEventArgs> >::operator()<winrt::TerminalApp::implementation::AppLogic,winrt::TerminalApp::implementation::SettingsLoadEventArgs>
0c TerminalApp!winrt::TerminalApp::implementation::AppLogic::ReloadSettings
0d TerminalApp!`winrt::TerminalApp::implementation::AppLogic::AppLogic'::`2'::<lambda_1>::operator()
0e TerminalApp!std::invoke
0f TerminalApp!std::_Apply_impl
10 TerminalApp!std::apply
11 TerminalApp!`ThrottledFunc<0>::_trailing_edge'::`3'::<lambda_1>::operator()
12 TerminalApp!winrt::impl::delegate<winrt::Windows::System::DispatcherQueueHandler,`ThrottledFunc<0>::_trailing_edge'::`3'::<lambda_1> >::Invoke
13 CoreMessaging!Windows::System::DispatcherQueue::DeferInvokeCallback
14 CoreMessaging!CFlat::SehSafe::Execute<<lambda_a81ff790741c2a62f2197c2561f5fe49> >
15 CoreMessaging!Microsoft::CoreUI::ActionCallback::ImportAdapter$
16 CoreMessaging!Microsoft::CoreUI::Dispatch::DeferredCall::Callback_Dispatch
17 CoreMessaging!Microsoft::CoreUI::Dispatch::DeferredCallDispatcher::Callback_OnDispatch
18 CoreMessaging!Microsoft::CoreUI::Dispatch::EventLoop::Callback_RunCoreLoop
19 CoreMessaging!Microsoft::CoreUI::Dispatch::UserAdapter::OnUserDispatch
1a CoreMessaging!Microsoft::CoreUI::Dispatch::UserAdapter::DoWork
1b CoreMessaging!Microsoft::CoreUI::Dispatch::UserAdapter::WindowProc
1c user32!UserCallWinProcCheckWow
1d user32!DispatchClientMessage
1e user32!_fnDWORD
1f ntdll!KiUserCallbackDispatcherContinue
20 win32u!NtUserGetMessage
21 user32!GetMessageW
22 WindowsTerminal!WindowEmperor::WaitForWindows
23 WindowsTerminal!WindowEmperor::HandleCommandlineArgs
24 WindowsTerminal!wWinMain
25 WindowsTerminal!invoke_main
26 WindowsTerminal!__scrt_common_main_seh
27 kernel32!BaseThreadInitThunk
28 ntdll!RtlUserThreadStart

Here's the exception:

0:000> .exr -1
ExceptionAddress: 00007ffb5165a29c (TerminalApp!winrt::impl::consume_Windows_UI_Core_ICoreDispatcher<winrt::Windows::UI::Core::ICoreDispatcher>::RunAsync+0x0000000000000054)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000001
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 0000000000000000
Attempt to read from address 0000000000000000

Here's the location of the crash: https://github.com/microsoft/terminal/blob/release-1.20/src/cascadia/TerminalApp/TerminalWindow.cpp#L773

In other words: It crashes because the settings have changed, which triggers a settings reload on all open windows. I believe this indicates that we either have a bug where when a window gets closed it doesn't get unregistered from the list of open windows, or we have a bug where it doesn't check whether a window is currently in the process of closing.

I'll hotfix this issue by checking if the thread dispatcher is a nullptr. That's not a good fix though, because there could be more similar bugs. Unfortunately, I believe that we cannot thoroughly fix this issue for the versions between v1.20 and v1.22. I'm hoping to do so in v1.23 by rewriting our windowing system. Hopefully, the hotfix will be enough.

microsoft / terminal