asherber / Xunit.Priority

Provides an ITestCaseOrderer that allows you to control the order of execution of Xunit tests within a class.
Apache License 2.0
49 stars 3 forks source link

Class Priority Control #3

Open JD-Howard opened 1 year ago

JD-Howard commented 1 year ago

I have a small automation app scenario where the bulk of my integration tests are all creating an artifact, validating the artifact and then that artifact gets used with a subsequent test. I realize that is horrible, but it is what makes sense for my situation. So, you can consider every class and every test within the classes prioritized.

Unfortunately, Xunit.Priority doesn't seem up to that level of rigid sequencing. It appears to only prioritize the tests within a class, not every test in the run. Know of a simple way of forcing X.P. to run tests in exactly the order they are specified and not allow parallel testing across classes?

I am probably going to ditch X.P. on this project and follow the below article, but it would be real nice if there was a simple XUnit based nuget package out there for accomplishing this. If you are interested in this enhancement request, note that the ideal solution would involve me being able to specify what can/cannot be parallelized. https://hamidmosalla.com/2018/08/16/xunit-control-the-test-execution-order/

asherber commented 1 year ago

It sounds like you're asking about a few different things, most of which have solutions.

For test parallelism in general, see the xUnit docs. By default, the tests within a class are run serially, although multiple classes may be run in parallel. The docs give you a few different ways to change this behavior, so that none or some of the classes can be run in parallel.

For running classes in serial in a specific order, the article you link to has the basic steps. You might also have luck with the Xunit.Extensions.Ordering NuGet package.

I obviously don't know much about your particular scenario, but have you considered combining tests from different classes into a single class, where they can all easily be ordered?