Possibility to specify test execution order

[r-weisser]

Hello, I want to implement some integration tests. How can I specifiy an execution order if the tests rely on each other like a test chain? Is there a sample how multiple classes acts on execution? I need to execute the different test classes in a specific order as well.

In the default implementation using fluent syntax the it["..."] tests are always executed before the nested context from the same level.

[BrainCrumbz]

Hi @r-weisser , glad to hear you're using NSpec.

Actually there's no way to order execution of tests (say, it blocks) within the same context. At least from experience in our team - and in general testing literature - relying on test execution order is generally discouraged.

We do have many cases though where a particular test makes sense only in a specific scenario, so you end up building a number of test scenarios with many paths and sub-paths:

• When this happens
  • Test this A
  • And when also this happens
  • Test this B1
  • Test this B2
  • And when finally this happens
    • Test this C
  • Instead when something else happens
  • Test this D1
  • Test this D2

Here, you know for sure that A test will run before B tests, which in turn will run before C test. Accordingly, you can run preparatory setup for each of those context, and those setup will run in order too. The only thing, you cannot tell the order between B1 and B2, for example.

If it fits your bill, NSpec excels at this test scenario construction, and it makes for a nice output of that as well.

[r-weisser]

Thanks for the fast response. I'll give it a try.

In the meantime I'm still wondering how the following setup will behave regarding order or parallization. Are they executed in one step - first class 1 then class 2 OR first class 2 then class 1 OR can the tests executed in a random mixed order like 1.1, 1.21, 2.1, 1.22, 2.22, ... ?

• Using multiple classes with tests e.g.
  • Class 1
    • Before
    • Test 1.1
    • Context 1.2
      • Test 1.2.1 executed first or second
      • Test 1.2.2 executed first or second
    • After
  • Class 2
    • Before
    • Test 2.1
    • Context 2.2
      • Test 2.2.1 executed first or second
      • Test 2.2.2 executed first or second
    • After

[BrainCrumbz]

There's currently no parallelization in NSpec test execution.

Regarding order of classes/contexts: within a test assembly, each nspec-derived class is executed one after the other, no order guaranteed. Within a spec class, each context "at the same level" is executed one after the other, again no order guaranteed. Before leaving a context to pass to the next of its siblings, all its sub-contexts are traversed and executed recursively.

Regarding order of before/act/test/after/etc., there are info about that in wiki as well as in main site. This is important as, for example, before hook from a context is run for each test under that context, so all in all it will run multiple times, while beforeAll is run just once for that context.

[amirrajan]

Relevant Stack Exchange Code Review QnA

Specifically this comment.