Result 'null' from combinatorial test run

[paulch42]

Consider the module

module ct
exports all
definitions 

functions 
  f: nat * nat -> nat
  f(m,n) == if m=n then m else f(m+1,n)
  post RESULT = n;

traces
  t_f: f(0,500);
end ct

When run from Combinatorial Testing view produces expected result. If the argument to f is increased such that stack overflow occurs due to recursion depth (the exact number will vary depending on JVM parameters), the Result null is reported in the CT Test Case Result tab, and no indication given concerning the error.

A more informative message would be useful to assist in tracking down the problem.

[peterwvj]

Again, thanks for describing the problem very well.

I updated the trace interpreter to report stack overflows

Consider a slightly different version of your spec:

module ct
exports all
definitions 

functions 
  f: nat * nat -> nat
  f(m,n) == if m=n then m else f(m+1,n)
  post RESULT = n;

  h: () -> nat
  h() == 1;

traces
  t_f: (h();f(0,10000);h());
end ct

The test results are now reported as

[paulch42]

Thanks for the quick action. This will certainly save time trying to track down why a test run fails.