Do not fail tests based on Timeout when debugging

microsoft / testfx

MSTest framework and adapter

MIT License

748 stars 255 forks source link

Do not fail tests based on Timeout when debugging #291

Open Tyrrrz opened 7 years ago

Tyrrrz commented 7 years ago

Description

It would be nice if the tests didn't fail due to timeout while debugging, just like in NUnit. Currently, you need to remove the attribute to properly debug such tests, because otherwise it's aborted abruptly in the middle of debug.

Steps to reproduce

Put [Timeout] attribute on a test method.
Launch the test method in debug mode and enter some breakpoint
Wait until the timeout is exceeded and then step over

Expected behavior

Debugging should continue without problems because the timeout is disabled during debug

Actual behavior

The test has stopped abruptly, not letting you finish debugging

Environment

Windows 10 vstest.console v15.0.26929.2 nuget packages v1.2.0

AbhitejJohn commented 7 years ago

I think that's a fair point. This could be really annoying.

dotMorten commented 6 years ago

I agree it's super annoying, however how do you debug a testrun where some tests are actually timing out? Ie you do a test run with the debugger attached, a test run could hang before you get to the one you were interested in debugging

Kissaki commented 4 months ago

Do you want/expect it to time only while the application is not paused, or never timeout indefinitely?

I would find a lack of timeout surprising and unexpected when the test specifically has a timeout specified.

nohwnd commented 4 months ago

Link added to the list.

dotMorten commented 4 months ago

I don't think just having the debugger attached should disable timeouts. If you do a full test run with the debugger attached, you could have a test hang indefinitely. IMHO it should only be pausing while you're at a breakpoint, but I don't quite think that's possible to do without some ugly timer loop that also gets paused instead of relying on timestamps.

nohwnd commented 4 months ago

I agree with you dotMorten, having the timer pause when the application is paused seems like the best solution, that way I can debug the code and still have it behave close to what it would behave like in real life.