Crashing when scrolling with touchpad but not mouse wheel

[themikem]

When scrolling terminal with a two-finger gesture on my laptop's touchpad, whether at a prompt or in settings, scrolling via the touchpad causes Terminal to instantly crash / exit / disappear, while scrolling with the mouse wheel does not and works as expected. Scrolling via touchscreen also works as expected.

Is there any way to capture logs or anything else that would be helpful in debugging this issue?

Thanks!

[zadjii-msft]

A feedback hub recording might work!

/feedback

[ghost]

Hi there!

Can you please send us feedback with the Feedback Hub with this issue? Make sure to click the "Start recording" button, then reproduce the issue before submitting the feedback. Once it's submitted, paste the link here so we can more easily find your crash information on the back end?

Thanks!

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[delenda-delenda]

Not OP but I have the same problem. I've submitted feedback using the steps above. While not critical, this bug is super annoying and results in a lot of lost sessions!

[zadjii-msft]

@delenda-delenda can you share the aka.ms link to your feedback here? It's nearly impossible to search the backend without that link 🙃

[delenda-delenda]

Hi - sure, it's https://aka.ms/AAitha2

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[zadjii-msft]

Weird, there are 9 conhost dumps there, but no Terminal dumps. Weird. Lemme see if I can't psychic debug where this code is. There's some special code we have to try and handle trackpad scrolling on a certain class of devices.

If I'm not mistaken, trying to use the trackpad to scroll the Settings UI pages doesn't work at all, correct?

[delenda-delenda]

Ah - I might have changed the Default Terminal Application to be "Windows Console Host" to try to mitigate the issue, could that be why you're seeing conhost dumps instead of Terminal dumps?

Yep, you're correct, scrolling with trackpad on the Settings UI also crashes Terminal.

[zadjii-msft]

Yep, you're correct, scrolling with trackpad on the Settings UI also crashes Terminal.

😨

That's uh, actually not what I suspected? I thought it would just, not work, not crash the whole application. That's possibly more troubling? What kind of PC & trackpad are you using? IIRC this code was to mitigate Synaptics driver issues, but it's been a few years since that code was written.

Maybe the .etl traces will have an error message. I'll check there.

[zadjii-msft]

Managed to find this coming from somewhere in windows.ui.xaml:

Error Code	Symbolic Name	Error Description	Header
0xc000027b	-1073741189	STATUS_STOWED_EXCEPTION	An application-internal exception has occurred.	ntstatus.h

Danggit. That's all I could get, there's some stowed XAML exception, but I can't get at an actual .dmp that might have that stowed exception in it.

We might have more luck if you try collecting the dump manually. @delenda-delenda Can you try following the steps in this post to set up automatic crash dumps? (For more info, see this link) If that works, then you should be able to automatically get a .dmp of the terminal when it crashes. Then, can you zip that dump up and send it to us, so we can investigate? Thanks!

---

\

It's curious to me that our normal tracelogging doesn't capture a XAML stowed exception on the way out - I feel like most our other exceptions to get WIL logged, but not that. Maybe we could add some sort of stowed exception handler to specifically log? I thought I had a branch for this at one point...

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[delenda-delenda]

Hi @zadjii-msft,

Sorry haven't had chance to get the crash dump sorted. I've got it now - how would you like me to send it over?

[zadjii-msft]

@delenda-delenda Kinda depends how you're comfortable sharing. It might have PII in it, definitely including whatever output was in the Terminal at the time. So there's a little tradeoff between "how easy it is to share" bs "how private I want to be"

You can email it to me (my email address is in my profile), with the usual caveats of security that accompany email. If you're cool with that, ping me here when you send it. A OneDrive (or similar) link would also work about the same.

For the more secure option: you can upload a .dmp to Feedback Hub, without making FH try to take the dump itself. If you just drag/drop the dmp to the "Attached" section of FH, it should upload it.

Make sure to share the aka.ms link that Feedback Hub generates with the "Share" button - it's impossible to find without that.

Admittedly, I've never had anyone try the "manually attach to feedback hub" option yet, so idk if that'll actually work 😅 Pick your poison!

[delenda-delenda]

Hi @zadjii-msft - I trust you, email sent 😄

[zadjii-msft]

WELP This very unfortunately looks like code that we (the Terminal team) can't fix. The stack firmly points to somewhere in XAML code.

Stack details

``` 0:000> k # Child-SP RetAddr Call Site 00 0000001e`d411e910 00007ffa`afd1e1f9 KERNELBASE!RaiseFailFastException+0x152 [minkernel\kernelbase\xcpt.c @ 1198] 01 0000001e`d411eef0 00007ffa`7ce6cf08 combase!RoFailFastWithErrorContextInternal2+0x4d9 [onecore\com\combase\winrt\error\error.cpp @ 1468] 02 0000001e`d411f160 00007ffa`7cc5a411 Windows_UI_Xaml!CJupiterWindow::OnIslandDirectManipulationHitTest+0x94 [onecoreuap\windows\dxaml\xcp\dxaml\lib\jupiterwindow.cpp @ 2281] 03 0000001e`d411f1f0 00007ffa`7cc59d2e Windows_UI_Xaml!CXamlIslandRoot::OnIslandDirectManipulationHitTest+0x71 [onecoreuap\windows\dxaml\xcp\core\core\elements\xamlislandroot.cpp @ 596] 04 (Inline Function) --------`-------- Windows_UI_Xaml!CXamlIslandRoot::SubscribeToPointerInputObserverEvents::__l91::::operator()+0x30 [onecoreuap\windows\dxaml\xcp\core\core\elements\xamlislandroot.cpp @ 1510] 05 0000001e`d411f220 00007ffa`a1eb2a47 Windows_UI_Xaml!Microsoft::WRL::Details::DelegateArgTraits >::*)(Windows::UI::Core::IInternalDirectManipulationInterop *,Windows::UI::Core::IPointerEventArgs *)>::DelegateInvokeHelper,Windows::Foundation::ITypedEventHandler,Microsoft::WRL::FtmBase>, &,-1,Windows::UI::Core::IInternalDirectManipulationInterop *,Windows::UI::Core::IPointerEventArgs *>::Invoke+0x3e [onecore\external\sdk\inc\wrl\event.h @ 354] 06 (Inline Function) --------`-------- InputHost!Microsoft::WRL::Details::CreateAgileHelper::__l2::::operator()+0x3d [onecore\external\sdk\inc\wrl\event.h @ 440] 07 0000001e`d411f250 00007ffa`a1eb1eff InputHost!Microsoft::WRL::Details::DelegateArgTraits >::*)(Windows::UI::Core::IInternalDirectManipulationInterop *,Windows::UI::Core::IPointerEventArgs *)>::DelegateInvokeHelper,Windows::Foundation::ITypedEventHandler,Microsoft::WRL::FtmBase>,,-1,Windows::UI::Core::IInternalDirectManipulationInterop *,Windows::UI::Core::IPointerEventArgs *>::Invoke+0x57 [onecore\external\sdk\inc\wrl\event.h @ 354] 08 (Inline Function) --------`-------- InputHost!Microsoft::WRL::EventSource,Microsoft::WRL::InvokeModeOptions<-2> >::InvokeAll::__l2::::operator()+0x28 [onecore\external\sdk\inc\wrl\event.h @ 964] 09 0000001e`d411f280 00007ffa`a1eb1db9 InputHost!Microsoft::WRL::InvokeTraits<-2>::InvokeDelegates<,Windows::Foundation::ITypedEventHandler >+0x6b [onecore\internal\sdk\inc\wrl\internalevent.h @ 121] 0a 0000001e`d411f2e0 00007ffa`a1eb2463 InputHost!Microsoft::WRL::EventSource,Microsoft::WRL::InvokeModeOptions<-2> >::DoInvoke< >+0x7d [onecore\external\sdk\inc\wrl\event.h @ 954] 0b (Inline Function) --------`-------- InputHost!Microsoft::WRL::EventSource,Microsoft::WRL::InvokeModeOptions<-2> >::InvokeAll+0x35 [onecore\external\sdk\inc\wrl\event.h @ 964] 0c (Inline Function) --------`-------- InputHost!PointerInputObserverWinRT::InvokeEventDirectlyHelper_Callback::__l2::::operator()+0x4d [onecoreuap\windows\moderncore\Inputv2\Components\AttachableInputObject\system\WindowsMessageDelivery\client\lib\PointerInputObserverWinRT.h @ 121] 0d 0000001e`d411f320 00007ffa`a1eb22b3 InputHost!Microsoft::WRL2::ContextSession::LeaveSession_Callback< >+0x77 [onecore\private\sdk\inc\wrl2\Wrl2Agile.h @ 1146] 0e (Inline Function) --------`-------- InputHost!PointerInputObserverWinRT::InvokeEventDirectlyHelper_Callback+0x21 [onecoreuap\windows\moderncore\Inputv2\Components\AttachableInputObject\system\WindowsMessageDelivery\client\lib\PointerInputObserverWinRT.h @ 119] 0f 0000001e`d411f380 00007ffa`a1e615e4 InputHost!PointerInputObserverWinRT::InvokeEventHelper_Callback,Microsoft::WRL::InvokeModeOptions<-2> >,PointerInputObserverWinRT::Partner>+0xb3 [onecoreuap\windows\moderncore\Inputv2\Components\AttachableInputObject\system\WindowsMessageDelivery\client\lib\PointerInputObserverWinRT.h @ 182] 10 (Inline Function) --------`-------- InputHost!PointerInputObserverWinRT::OnDmPointerHitTestMessage_Callback+0x22 [onecoreuap\windows\moderncore\inputv2\components\attachableinputobject\system\windowsmessagedelivery\client\lib\pointerinputobserverwinrt.cpp @ 605] 11 0000001e`d411f400 00007ffa`a1e085ab InputHost!PointerInputObserverWinRT::OnWindowMessage_Callback+0x318c4 [onecoreuap\windows\moderncore\inputv2\components\attachableinputobject\system\windowsmessagedelivery\client\lib\pointerinputobserverwinrt.cpp @ 403] 12 (Inline Function) --------`-------- InputHost!WindowsMessageDeliveryInputSiteWinRT::OnWindowMessage_Callback::__l2::::operator()+0x69 [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\windowsmessagedeliveryinputsitewinrt.cpp @ 142] 13 (Inline Function) --------`-------- InputHost!std::invoke+0x69 [onecore\internal\sdk\inc\ucrt\type_traits @ 1488] 14 (Inline Function) --------`-------- InputHost!std::_Invoker_ret::_Call+0x69 [onecore\internal\sdk\inc\ucrt\stl120\functional @ 660] 15 0000001e`d411f430 00007ffa`a1e07c97 InputHost!std::_Func_impl_no_alloc<,enum InputSiteIterationDecision,Microsoft::WRL::ComPtr const &>::_Do_call+0x7b [onecore\internal\sdk\inc\ucrt\stl120\functional @ 822] 16 (Inline Function) --------`-------- InputHost!std::_Func_class const &>::operator()+0x27 [onecore\internal\sdk\inc\ucrt\stl120\functional @ 869] 17 0000001e`d411f460 00007ffa`a1e06d29 InputHost!InputSiteWinRT::ForEachAttachedObject+0x277 [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\inputsitewinrt.cpp @ 138] 18 (Inline Function) --------`-------- InputHost!WindowsMessageDeliveryInputSiteWinRT::OnWindowMessage_Callback+0x6a [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\windowsmessagedeliveryinputsitewinrt.cpp @ 137] 19 0000001e`d411f510 00007ffa`a1e2de33 InputHost!SystemIslandInputSiteWinRT::OnWindowMessage_Callback+0xa9 [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\islandinputsitewinrt.cpp @ 303] 1a 0000001e`d411f5e0 00007ffa`a1e06f69 InputHost!WindowsMessageDeliveryInputSiteWinRT::Private::OnWindowMessage_Callback+0x63 [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\windowsmessagedeliveryinputsitewinrt.cpp @ 262] 1b (Inline Function) --------`-------- InputHost!WindowsMessageDeliveryAdapter::ProcessWindowMessage_NoLock+0x13d [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\windowsmessagedeliveryadapter.cpp @ 325] 1c 0000001e`d411f680 00007ffa`afe98161 InputHost!WindowsMessageDeliveryAdapter::StaticWndProc+0x179 [onecoreuap\windows\moderncore\inputv2\components\inputsitemanager\client\lib\system\windowsmessagedeliveryadapter.cpp @ 257] 1d 0000001e`d411f7b0 00007ffa`afe97e1c user32!UserCallWinProcCheckWow+0x2d1 [clientcore\windows\core\ntuser\client\clmsg.cxx @ 282] 1e 0000001e`d411f910 00007ffa`afea2cdd user32!DispatchClientMessage+0x9c [clientcore\windows\core\ntuser\client\daytona\objfre\amd64\client.cxx @ 3113] 1f 0000001e`d411f970 00007ffa`b0b52e44 user32!__fnDWORD+0x3d [onecoreuap\restricted\windows\inc\ntuser\inc\ntcb.h @ 1214] 20 0000001e`d411f9d0 00007ffa`ae881534 ntdll!KiUserCallbackDispatcherContinue [minkernel\ntos\rtl\amd64\trampoln.asm @ 607] 21 0000001e`d411fa58 00007ffa`afea500a win32u!ZwUserGetMessage+0x14 [onecoreuap\windows\core\umode\moderncore\objfre\amd64\usrstubs.asm @ 189] 22 0000001e`d411fa60 00007ff6`7b6979c7 user32!GetMessageW+0x2a [onecoreuap\restricted\windows\inc\private\core\ntuser\client\ntcftxt.h @ 537] 23 0000001e`d411fac0 00000000`00000001 WindowsTerminal!wWinMain+0x1a7 [C:\a\_work\1\s\src\cascadia\WindowsTerminal\main.cpp @ 149] FAULTING_SOURCE_CODE: 2277: 2278: MarkMessageHandlerWrapper markMessageHandler(m_pControl); 2279: 2280: wrl::ComPtr pointerPoint; > 2281: IFCFAILFAST(args->get_CurrentPoint(&pointerPoint)); 2282: 2283: UINT32 pointerId = 0; 2284: IFCFAILFAST(pointerPoint->get_PointerId(&pointerId)); 2285: 2286: wrl::ComPtr coreWindowEventArgs; ```

You're on 10.0.22621.1, so I'm worried this might be a recent regression in Xaml Islands. Lemme dig to see if there's an internal bug with this.

[zadjii-msft]

Alrighty, I promoted this to MSFT:42633157 internally. Hopefully after the holidays, someone on the WinUI team can take a look.

[delenda-delenda]

Interestingly, I'm getting the same issue when using the trackpad to scroll with the Phone Link app :(

[zadjii-msft]

Yep, that pretty much confirms my theory that this is a WinUI issue, not a Terminal one.

[rossnichols]

@delenda-delenda @themikem hi, I'm another engineer at Microsoft. Would you mind telling me what sort of touchpad you're using to scroll? If built into your laptop, what model laptop? If an external device, what is it?

[darktears]

@rossnichols I have the same issue reported here. I'm using the touchpad of the laptop, two fingers scrolling. It has been crashing since I received the laptop. I can scroll just fine with an externally connected mouse, using my fingers on the touch screen or using the scrollbar with the cursor. Now when I use the touchpad it crashes right away. I reported the issue with the feedback hub, the dumps should be there : https://1drv.ms/u/s!AqJx-UJTdDLhpIMb1JtGPp7GPrL7MA?e=xQxvba

Laptop: ASUS Zenbook 17 Fold OLED UX9702AA Device Manager regarding the mouse: Default HID compliant mouse on Bluetooth Low Energy GATT compliant HID device. PNP Device ID: HID{00001812-0000-1000-800000805F9B34FB}_DEV_VID&020B05_PID&1A44_REV&0B05_C419D1A4DC1A&COL02\9&219F2D6B&0&0001 OS : Windows 11 Enterprise, 22H2, 22621.1992, Windows Feature Experience Pack 1000.22644.1000.0

[darktears]

Attaching Visual Studio to the Terminal I can reproduce the same stack trace than above: https://gist.github.com/darktears/0c5e03335d6ffeca7bf8931798942d02

[rossnichols]

@darktears do other apps crash when you scroll them, such as Settings?

[darktears]

Nope, they work just fine, I don't think Asus would have let this slipped.

[rossnichols]

@darktears can you run this program and go to View => Digitizer Capabilities, expand the touchpad node and all its children, and take screenshots of the values? Sorry it's a bit cumbersome, the tool wasn't designed to easily export this info, just view it :(. Actually the tool does have an export, it's just not quite up to this task :(

https://microsoft-my.sharepoint-df.com/:f:/p/rnichols/Ek1ma50Ot6JOlVMVinBzT1kBuGY_lrrtiODY1KCFDHitqA?e=Wi7ytl

[darktears]

Let me know if you want something specific in the tree expanded.

[rossnichols]

@darktears can you take screenshots of the "Usage Vendor Defined UsagePage (26)" node, as well as (27)? And can you confirm that the following all have LinkCollection = 0?

These input values:

• SCANTIME
• ACTUALCOUNT
• Unknown usage in BUTTON

This input button:

• BUTTON1

[darktears]

They all have LinkCollection set to 0.

[rossnichols]

The root cause of this issue is the touchpad's HID descriptor containing different data across the device's child collections representing parallel-reporting fingers. Per https://learn.microsoft.com/en-us/windows-hardware/design/component-guidelines/selecting-packet-reporting-modes-in-multitouch-devices, "[w]hen taking advantage of Parallel mode, each of the logical collections must be identical". This touchpad's parallel logical collections differ in their vendor-defined data - the same usage page (0xFF01), but different usages (26, 27, etc.).

Unfortunately, the Windows OS does not enforce this requirement when the device is enumerated, so it still loads and can function essentially normally. However, the rule-break impacts GetPointerDeviceProperties and GetRawPointerDeviceData in such a way that makes the API fail, leading to crashes in UI frameworks not resilient to failures from these APIs, which should not fail in normal conditions.

The correct fix here would be from the touchpad manufacturer, to update the device's HID descriptor to obey the requirements. However, since these devices are already present in the ecosystem, I'll investigate how Windows can be more resilient to this rule being broken by a device.

[zadjii-msft]

(internal notes: MSFT:42633157 should be fixed by os.2020!9381629. No ETA on when that'll actually ship, but Soon^TM)

Bug thanks to @rossnichols for digging in and fixing this on the OS side ☺️ I'm gonna close this one out on our repo, since I don't think there's anything left for the Terminal team to do here.

---

update: !9381629 merged yesterday, oct 26 2023. Probably won't get to Insiders for 4+ weeks, but the fix is on it's way

[mangini]

@darktears can you run this program and go to View => Digitizer Capabilities, expand the touchpad node and all its children, and take screenshots of the values? Sorry it's a bit cumbersome, the tool wasn't designed to easily export this info, just view it :(. Actually the tool does have an export, it's just not quite up to this task :(

https://microsoft-my.sharepoint-df.com/:f:/p/rnichols/Ek1ma50Ot6JOlVMVinBzT1kBuGY_lrrtiODY1KCFDHitqA?e=Wi7ytl

@rossnichols we are trying to debug a crash in our app that is very similar to the one described in this bug. The tool above is not available anymore. Do you have a link to it or to its source code? Thank you!

[rossnichols]

@mangini here it is again: https://1drv.ms/f/s!At5lGHRE4ciEm_ZVFVSiqgEtA-hKHQ?e=jsafTX

This should be fixed in the Windows 11 24H2 release, if you can still repro there, please let me know. I can look into backporting the fix to Windows 11 22H2 as well if necessary, but it's not planned at this time.

[mangini]

Appreciate the quick response @rossnichols . Unfortunately we don't control what version of the OS our users have. Do you know if there is anything we can do on the software side that could work around the crash or avoid it at the application side? We use Windows App SDK and if necessary we can detour some public APIs.

We don't know for sure if it is the same bug, but it certainly looks very similar, and it is currently our top crash. Early next week I'll check with this user who can trigger the crash very frequently and see if his touchpad has an invalid HID descriptor.

[darktears]

I would also point out that as of today I don’t have the fix as well. I was hoping that it would have made it to an earlier version of Windows. 24H2 means another 6+ months at least for the corporate world.

[rossnichols]

@darktears fortunately in between when I made the original fix and now, it has become easier to bring these sorts changes back to 22H2 - I'll take a look.

@mangini can you give more details about the call stack of your crash and the variety of devices that hits it? To date we're only aware of a single device model that hits this crash - the Asus Zenbook 17 Fold.

[mangini]

@rossnichols : We get a very generic stack trace from those crashes. In the minidump, it crashes on onecoreuap\windows\moderncore\inputv2\systeminputhosts\lifted\lib\delegatedmasterinputthread.cpp:223 (for WinApp SDK 1.5.240627000)

And here's the stack that Sentry collects:

Here's the result of our engineer running your tool on his laptop, a Surface Laptop 5 (Core i5, 16G RAM) running Windows 11 Pro (23H2). This is the only internal employee who has this crash regularly, almost daily, but we have hundreds of external users reporting this crash every week.

[rossnichols]

@mangini this seems unrelated. I believe I found the internal bug for it and it's fixed as of 1.5.6 - can you please try upgrading?

[mangini]

Thanks @rossnichols ! We also expected it to be fixed in 1.5.6 - that's what our MS rep told us - but unfortunately it did not. We upgraded to 1.5.6 two releases ago (we release weekly) and the frequency of this crash remains the same unfortunately.

One of our engineers pointed out that we are not calling ContentPreTranslateMessage in our message pump, unlike what this xaml example does without much explanation: https://github.com/microsoft/microsoft-ui-xaml/blob/137c8d6ce57e68f88aea35d8131fb71248b09c6c/src/controls/test/TabViewTearOutApp/DesktopWindow.cpp#L149

However, since this method is not documented, we are not sure if we should. Do you think this could help?

Also, do you believe this can be tied to specific laptop (or touchpad) models? We don't collect computer brand/model from users for privacy reasons, but if there is anything we could collect that has no privacy implications - for example specific HID configurations, we can set it up.

[rossnichols]

@mangini isn't this its documentation? I believe it's important to call for proper Xaml interop, but it isn't related to this crash.

I pinged a coworker and he said that it's a known issue and the fix should be in 1.5.7 - please let me know if that upgrade does not work, when it's released. Thanks for your patience!

[rossnichols]

@mangini a clarification - the crash in CheckDelegatedInputQueue should have been fixed in 1.5.6, but there was a regression from the fix that led to a new crash that is being fixed in 1.5.7. Can you confirm the call stack of the crashes you're seeing on 1.5.6?

[mangini]

I pinged a coworker and he said that it's a known issue and the fix should be in 1.5.7 - please let me know if that upgrade does not work, when it's released. Thanks for your patience!

Interesting, thank you a ton for checking. Is there any other information about the fix, or anything we can try to do on the application side to work around it? We could try to detour public methods if we know why is is happening.

Do you have a build of 1.5.7 that we could test internally with this user?

Re the call stack in 1.5.6, it seems identical to the previous version:

[rossnichols]

@mangini can you confirm the exact version string of WinAppSDK that you're using?

[mangini]

<?xml version='1.0' encoding='utf-8'?>
<packages>
  <package id="Microsoft.WindowsAppSDK" version="1.5.240802000" />
 ...

[rossnichols]

@mangini thanks - is it possible to share a crash dump? What is your engineer doing when it repros?

[mangini]

Sorry for the delay in responding. I had to set up a minidump collection. Could you send me (my github name at gmail) an address where I can send the link to the dmp? (I have the slimmest dump, but just in case there is any PII in the stack)

What is your engineer doing when it repros?

All the times he could remember, he was scrolling the app with the touchpad, using two fingers. Both vertical and horizontal scrolling. He has this crash almost daily.