compile certain test functions differently

[jurgenvinju]

Proposal for enhancement of test functions. Particularly the specification-style functions with random parameters could benefit from a little online diagnostics.

Take this example:

test myTest(int x, int y) = f(x) == g(y);

apparently in theory the output of f(x) should equal the output of g(y). There exist many tests which have an equation as top operator. For these functions I'd like it if we generate code which looks like this:

test myTest(int x, int y) {
  lhs = f(x);
  rhs = g(x);
  if (lhs != rhs) {
    println("FAILURE: <lhs> != <rhs>, because of these differences:
            '<diff(lhs, rhs)>");
    return false;
  }
  return true;
}

Same test semantics, but as a side-effect it explains and reports the differences immediately.

[jurgenvinju]

for complex or high test parameters, an attempt at input minimization would also be appreciated (before reporting the big data structure differences).

[DavyLandman]

Currenty the random tester in case of an error, tries to find the smallest set of arguments of the random generator (width & depth) to also fail that test. It does not however report the original input, if it would find a smaller case (that indeed, might be a different bug).

[jurgenvinju]

Does it also reduce parse trees? and ASTs? I do that in DrAmbiguity (while the bug remains try 100 times to randomly reduce the tree by removing elements from lists, contracting recursive rule applications, removing optionals, etc. etc.)