Order of test case execution

[Garados007]

I found something interesting when I took some preparations for #375 and I am not sure if it's intended or not. Let's say we have two procedure files Proc0 and Proc1 with mostly the same code:

#pragma TextEncoding = "UTF-8"
#pragma rtGlobals=3             // Use modern global access method and strict wave access
#pragma DefaultTab={3,20,4}     // Set default tab width in Igor Pro 9 and later
#pragma ModuleName=Proc0 // procedure Proc1 has the module name Proc1

#include "igortest"

static Function TestA()
    PASS()
End

static Function TestB()
    PASS()
End

If I execute RunTest("Proc0;Proc1", testcase="TestA;TestB") I get the following TestRunData:

I would have expected that IUTF would execute all test cases from Proc0 first and then Proc1 but it seems like the test cases are executed out of order.

I don't know it this is intended behavior so I am asking for it.

[t-b]

I think this is a bug. @MichaelHuth What do you think?

[MichaelHuth]

I think that we never specified the order preference of modules. Regarding our test case order specification based on the simple line numbers of function names it is correct. But considering the full function name, I would agree that we should sort by module name with preference and then by function name line number. I think the fix in SortTestCaseList shouldn't be too hard as it jsut need an extension to Sort with a primary + secondary key wave.

[t-b]

@MichaelHuth Thanks.

I would have expected that IUTF would execute all test cases from Proc0 first and then Proc1 but it seems like the test cases are executed out of order.

I would think that it should be like that as well. If we do that with sorting we have to use the user supplied list of testsuites if present as

RunTest("Proc0;Proc1", testcase="TestA;TestB")

should do Proc0 and then Proc1

but

RunTest("Proc1;Proc0", testcase="TestA;TestB")

should do Proc1 and then Proc0.