search

microsoft / vstest

Visual Studio Test Platform is the runner and engine that powers test explorer and vstest.console.
MIT License
883 stars 319 forks source link

Testhost is blocked on system error message dialog #5151

Closed riQQ closed 1 month ago

riQQ commented 1 month ago

Description

The testhost is blocked on a system error message dialog. I haven't analyzed the cause of the system error yet, but the problem here is that a system error dialog is shown at all. Running the tests in a build pipeline via Azure DevOps / VSTest task / vstest.console the task just hangs until the timeout is hit. This issue can only be diagnosed with access to the CI system to see that it's the system error dialog that causes the timeout.

Steps to reproduce

So far I haven't managed to produce a small sample that I can share. I'll update this post if I manage to produce a self contained repro.

Expected behavior

Testhost crashes and the error is logged.

Actual behavior

Testhost is blocked on system error message dialog

Diagnostic logs

There is no mention of the error in the logs, just polling on both sides.

vstest.console console output after clicking OK on the system error dialog

The active test run was aborted. Reason: Test host process crashed

vstest.console diagnostic log

TpTrace Verbose: 0 : 26720, 10, 2024/07/18, 14:01:59.835, 180619017190, vstest.console, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:61110 localEndPoint: 127.0.0.1:61109 after 1006 ms

testhost

TpTrace Verbose: 0 : 13916, 11, 2024/07/18, 14:04:31.099, 182131653244, testhost.net472.exe, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:61109 localEndPoint: 127.0.0.1:61110 after 1013 ms

Environment

Windows 10 VSTest from Visual Studio installation testhost.net472.exe / testhost.net472.x86.exe version 17.10.0-release-24177-07 vstest.console.exe 17.10.0-release-24177-07 ()

nohwnd commented 1 month ago

I don't think there is a way for us to influence this. It is the same deal with debug assertion messages (Debug.Assert). Unless the underlying framework implements a trace listener that does not forward the assert failure, the default will show an assertion pop-up, and will block the run.

Is it happening in any run? My impression is that when the session is not interactive you don't get these pop-ups, and imho interactive session is set only when you say you have UI tests in the mix (at least when using VSTest AzDO task).

nohwnd commented 1 month ago

We most likely cannot affect this, and it would be a new feature, or more likely a responsibility of the testing framework (e.g. mstest, nunit, xunit) rather than responsibility of this platform.

I would be curious if you come up with a solution, if yes we could consider it for testing platform / mstest (both residing in https://github.com/microsoft/testfx). 😊

But at the moment this repo (vstest) is not accepting any new features.