search

dotnet / project-system

The .NET Project System for Visual Studio
MIT License
959 stars 385 forks source link

Increase concurrency of unit tests #9451

Closed drewnoakes closed 2 months ago

drewnoakes commented 2 months ago

xUnit runs all tests within a "collection" concurrently. However the default behaviour is to have one collection per test class. This commit changes the default to be one collection per assembly (unless a specific collection is defined). This means that, but default, most tests in the assembly will now be able to run concurrently.

It's possible that this introduces test instability, though if generally we shouldn't have tests that require isolation from other parts of the system. We will monitor for such instability, and also check whether this change improves test execution times.

Microsoft Reviewers: Open in CodeFlow
drewnoakes commented 2 months ago

It's possible that this introduces test instability

Seems to have worked on CI.

check whether this change improves test execution times

It doesn't seem to have helped much with CI builds. They generally take about 1 minute to complete, so there's no so much to gain there. I'll experiment locally to see whether this setting helps in the VS test runner.

smitpatel commented 2 months ago

I am not sure how making collection per assembly would increase test concurrency. It would make it actually more sequential.