microsoft / testfx

MSTest framework and adapter
MIT License
716 stars 253 forks source link

Do not rely on testhost to define thread apartment #1674

Closed Evangelink closed 5 months ago

Evangelink commented 1 year ago

Summary

Provide a way for users to specify that the given test(s) should be run on STA. This is particularly helpful for some UI testing with WPF, WinForms or some other client technologies.

Background and Motivation

Today, MSTest relies on the threading apartment of the hosting testhost to be STA or MTA. As shown in this issue https://github.com/microsoft/vstest/issues/1865 vstest creates a STA testhost for .NET Framework but not for the other frameworks.

This means that today, users who are targeting .NET and want to run their tests with STA have no mean to do so.

Proposed Feature

We could implement something similar to what is done by Andrew Arnott for xUnit (see https://github.com/AArnott/Xunit.StaFact) or try to find a way to only ask for users to define the [STAThread] attribute on their test and detect the available synchronisation context.

Alternative Designs

N/A

AB#2020496

### Tasks
- [x] MSTest runner should support running tests in STA
- [x] MSTest .NET Core through VSTest should support running tests in STA
fforjan commented 1 year ago

@Evangelink in such context, i'm always asking the question... what am i doing wrong because I wasnt expecting to be the first one to run 'UI' tests in .NET ? does it mean either no does testing in .NET or no one uses mstest for this, including teams at Microsoft ?

Evangelink commented 1 year ago

I'd love to have a good answer to provide you but I don't...

I am inheriting MSTest codebase and have to admit I wasn't using it prior to joining Microsoft. I am guessing many users are still using MSTest with .NET Framework and so this problem hasn't yet unveiled. I am pretty surprised at the number of differences (in features or behavior) between MSTest for .NET Framework and .NETCore

fforjan commented 1 year ago

Dear @Evangelink any update ?

Evangelink commented 1 year ago

No news on that front. We are having discussions about roadmap and future of some of the testing tools lately. I will keep you posted when I have some news.

fforjan commented 11 months ago

@Evangelink Désolé but I'm back to bother you again.

We are a bit blocked in our migration to .net core. This is a long journey and I just want to explain why this issue - with no workaround- is blocking us. We are looking to have a long .net core migration (multi year) due to the different technologies and size of the applications we are using. But overall, for each components, we do as such: 1- enable to compile the project and its unit tests for both .net framework and .net 6 2- enable unit tests to run both .net framework and .net 6 as part of our regular regression. At this stage, even if the application is NOT fully migrated and working as some components are not migrated, it allows us to ensure we are going the right way and not introducing code which could be broken on .net 6

With the current issue, we are NOT able to enable our unit tests relying on .WPF for .NET 6 and thus we have no 'view' about how our WPF control may work (or not) on net 6. There is so many moving part when looking at migrated from framework to .net 6 that it makes us - as a team- not comfortable.

Although we understand it may be complicated to have a proper fix, are you aware of any workaround or any options other than moving to xUnit to enable us on our .NET 6 journey ?

Evangelink commented 11 months ago

Hi @fforjan,

Sorry for the delay. I just had confirmation that there is no work planned on MSTest for the upcoming quarter. I cannot comment on the next period :c

Evangelink commented 9 months ago

Quick update to say that we will be fixing this issue in the next major of MSTest that should happen next year.

fforjan commented 9 months ago

@Evangelink Sorry for being pushy but happy we finally have a plan :)

Evangelink commented 5 months ago

@fforjan Sorry it took so long, I just wanted to let you know this is planned for the current sprint so the fix should be available in about a month.

fforjan commented 5 months ago

awesome !

testplatform-bot commented 5 months ago

✅ Successfully linked to Azure Boards work item(s):

Evangelink commented 5 months ago

Closing as done by #2682