orc-lang / orc

Orc programming language implementation
https://orc.csres.utexas.edu/
BSD 3-Clause "New" or "Revised" License
40 stars 3 forks source link

Allow specification of unordered correct output in test programs #51

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
The Orc test suite runs programs in the examples/ directory and compares their 
output to an 
OUTPUT: comment in the same file. This output comparison is sensitive to the 
order of the entries 
in the comment. 

For many programs, this is not appropriate, since nondeterministic evaluation 
could yield equally 
valid output in a different order. In particular, the dfs.orc test will 
occasionally report a failure 
simply because its output arrives in an order different from the expected order.

We would like to have options for both ordered and unordered output specs. A 
more expressive 
spec format would be even better, but is probably beyond the scope of this 
testing method.

Original issue reported on code.google.com by dkitc...@gmail.com on 22 Feb 2010 at 8:26

GoogleCodeExporter commented 9 years ago
Fixed in r1886

Original comment by jthywissen on 9 Aug 2010 at 10:56