wolverine2k / crunchy

Automatically exported from code.google.com/p/crunchy
0 stars 0 forks source link

Allow all_tests.py to run the tests in a fixed order without shuffling the files #196

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
In all_tests.py, the list of files is shuffled randomly to ensure the tests 
are run independently of one another. However, for tracking down ordinary 
test failures, it'd be useful to have all_tests.py run the tests in the 
same order between executions.

The attached patch adds a --det (meaning "deterministic") command-line 
option that skips the shuffling when it's passed. (I'm using 
optparse.OptionParser since it looks like from the todo comments that 
future work on all_tests.py will eventually require it anyway.) 
OptionParser was added to the standard library in Python 2.3, if that's 
relevant.

Current output
==============
> python all_tests.py         
0 failures in 26 tests in file: test_account_manager.rst
0 failures in 13 tests in file: test_execution.rst
0 failures in 59 tests in file: test_dhtml.rst
[snip]

> python all_tests.py
0 failures in 4 tests in file: test_errors.rst
0 failures in 56 tests in file: test_io_widget.rst
0 failures in 29 tests in file: test_menu.rst
[snip]

Expected output after patch
===========================
> python all_tests.py --det
0 failures in 26 tests in file: test_account_manager.rst
0 failures in 13 tests in file: test_comet.rst
0 failures in 19 tests in file: test_configuration.rst
[snip]

> python all_tests.py --det
0 failures in 26 tests in file: test_account_manager.rst
0 failures in 13 tests in file: test_comet.rst
0 failures in 19 tests in file: test_configuration.rst
[snip]

This wasn't a Python 3-specific issue so I figured I'd file a patch against 
the trunk.

Original issue reported on code.google.com by shadytr...@gmail.com on 29 May 2009 at 2:50

Attachments:

GoogleCodeExporter commented 8 years ago
This has now been taken care of as part of r1181.

Original comment by andre.ro...@gmail.com on 25 Jul 2009 at 11:37