microsoft / testfx

MSTest framework and adapter
MIT License
702 stars 251 forks source link

Timeout features should allow for a cooperative only mode #2432

Closed Evangelink closed 1 month ago

Evangelink commented 6 months ago

In the current design of the timeout feature, the methods marked with, and supporting, the timeout attribute when timeout expires will produce the following behaviors:

  1. the cancellation token linked to the related TestContext will be triggered
  2. the activity (thread or task for current implementation) of running the method will be unobserved resulting in a dangling activity.

The first part, cooperative cancellation, is virtuous but the second part, dangling activity, is not recommended and could lead to more problems (from CPU/Memory usage for nothing to global test app/suite being in some unknown state).

We cannot simply break the second behavior has it would required many changes from users but we should encourage the new behavior. To do so, I suggest the following course of action:

  1. In release v3.4 of MSTest, we will:

    • introduce a new property on timeout attribute to allow/disallow non-cooperative mode on the given method
    • introduce a runsettings entry to allow/disallow non-cooperative mode globally (the timeout property would override the global setting)
    • introduce an analyzer that flags cases where the non-cooperative mode is allowed
  2. In release v4 of MSTest, we will do a breaking change disallowing the non-cooperative mode by default.

AB#2034605

testplatform-bot commented 4 months ago

✅ Successfully linked to Azure Boards work item(s):