divinum-replay: add options to save a list of failed tests and to read tests to run from a file

When using divinum-replay.pl with a large set of test cases, it is useful to be 
able to save a list of tests that failed for further analysis. In order to 
perform this analysis, it would be useful to be able to take a list of tests to 
run from a file. The attached patch adds two switches for this purpose:

--failures=FILENAME   Write to FILENAME a list of files of tests that failed
                      (one test filename per line). If no test fails, an
                      empty file will be written.
--tests=FILENAME      Read names of test(s) from FILENAME (one test filename
                      per line). These tests will be added to any that are specified
                      directly on the command line. This flag can be repeated.

Example use:

1. Run a large test suite against a server and save a list of the failed tests 
to /tmp/failures:
    admin/divinum-replay.pl cases/base/* --failures=/tmp/failures

2. Re-run only the failed tests to see which differ in more than accents and 
punctuation:
    admin/divinum-replay.pl --tests=/tmp/failures --filter=-punctuation,-accents

3. If there are still a large number of failures, it may be useful to pipe the 
results through sort -u (on Unix-compatible systems, including OS X):
    admin/divinum-replay.pl --tests=/tmp/failures --filter=-punctuation,-accents | sort -u

4. After reviewing the failed tests to make sure there are no unexpected 
changes, update the cases (from the production server):
    admin/divinum-replay.pl --tests=/tmp/failures --update

Besides the patch, I also attach the complete updated script.

Original issue reported on code.google.com by a...@liturgiaetmusica.com on 6 Mar 2012 at 9:51

Attachments:

veromary / divinum-officium

divinum-replay: add options to save a list of failed tests and to read tests to run from a file #155